ww/examples/lisp/CLAUDE.md

examples/lisp — tiny Lisp interpreter in pure ww. Demo program; not a
production interpreter. Drives the wwstage cgen (`out/bin/w6c_ww`) by
default; the Makefile sets `WW_W6C` so `make` picks the ww-built
backend rather than the C bootstrap one.

## Layout

- `lispcore/`       interpreter directory package: types, lexer, parser, env,
                    eval, apply, printer, REPL. Everything the entry
                    point and the test driver consume is `export`-ed.
- `lisp.ww`         entry point; `use lispcore;` + `main()`.
- `lisp_test.ww`    in-process test driver (101 probes). Built as a
                    standalone binary, exec'd directly — `ww test`
                    drops `-I` in single-file mode, so the Makefile
                    runs the binary itself.
- `test_*.lisp`     demo source files (`cat test_X.lisp | ./lisp`).

The lispcore/ + lisp.ww split exists so the entry point and tests import
one canonical interpreter directory package. Cross-module type prefixes
like `lispcore.value` don't resolve in the test — use the bare names.

## wwstage cgen workarounds at play

The wwstage cgen has been brought up to par on the bug classes this
demo originally exercised. The remaining workarounds and the
historical bugs they sidestepped are listed below; "(retired)" marks
items the cgen now handles natively, kept here only as a record of
the shape so a regression is easy to recognise.

1. **(retired) Top-level `[N]T` arrays mis-address inside functions.**
   `globalarr[i]` used to lower to `LEAQ (BP), BX`. cgindex/cgassign
   now detect top-level array idents and emit `LEAQ name(SB), BX`,
   and emitletdataw lays the array bytes into DATAW.

2. **(retired) `global_ptr[i]` mis-addresses.** cgindex/cgassign
   detect top-level `*T` idents and emit `MOVQ name(SB), BX` with
   the correct element scaling. `lispcore`'s symbol interner is back
   to plain `sym_blob[off + i]` style; no `let blob: *u8 = sym_blob`
   aliasing.

3. **(retired) Two-level field write through a non-pointer
   sub-struct.** cgassign/cgdot now handle the chained
   `(*L).cur.kind` shape both as read and write. The lexer keeps the
   flattened `curkind` / `curival` / … fields for now because every
   call site uses them; un-flattening is a stylistic improvement, not
   a correctness fix.

4. **(partially retired) f64 through every boundary.** Struct-field
   `p.fval = v` through `*T`, `*p = v` for `*f64`, and `alloc(T{…})`
   sugar for f64 fields all route through X0 now (`vfloat`,
   `promote_v`, `to_f64` are back to direct `p.fval = …` /
   `*out = v.fval` form). Function-arg passing of an f64 struct-field
   value also works (the cgen's `exprfloatkind` now recognises
   `p.field` whose declared type is f64/f32).

5. **(retired) `alloc(value{ text = s })` writes only `s.ptr`.** Both
   the `alloc(T{…})` builtin and the bare struct-literal init now
   emit both halves of the str. `vstr` still does manual
   `p.text = s` (semantically equivalent, no longer required).

6. **(retired) `(slice | E)` tagged-union returns drop `slice.len`.**
   The return ABI now uses 4 regs (AX=tag, DX=ptr, CX=len, R8=cap),
   covering slice-payload variants up to 32B. The inline args-eval
   in `eval` is still there but no longer required — `apply` could
   factor it back out into a `(slice | rterror)` helper. Left as a
   readability cleanup; not a correctness fix.

7. **(retired) `xs[i].kind` drops the trailing field load.** cgdot
   now handles N_INDEX bases. Builtins are back to
   `xs[0].kind` / `xs[0].car` directly — no `let p = xs[0];` first.

8. **f64 compound assigns are mis-lowered to `acc = d` (no OP).**
   Still present. Write the explicit form: `acc = acc + f`. Integer
   compound assigns work fine.

9. **(retired in practice) `let r = call(); foreign_call(); return
   r?;` corrupts `r`.** The wwstage cgen now spills the AX/DX/CX
   triple to the local's 24-byte slot at the assignment point,
   matching what cstage emits — the foreign call in between no longer
   clobbers an unspilled half. We still pre-match the union inline in
   eval's BUILTIN apply path so the `os.free` happens *after*
   classification rather than after the unwrap.

If a new function "should work but acts weird", check #6 first.

## Tail-call optimization

`eval` is a single `for(true)` trampoline; a tail position rewrites
`v` (current expr) and `cure` (current env) in place and `continue`s
instead of recursing. Tail positions are:

- the chosen branch of `if`
- the last form of `begin` / `let` / a lambda body
- a direct call in any of the above

A `tailed` flag flips on the first jump into a fresh scope (LET
binding, lambda body) so subsequent `define`/`set!` mutate the local
`cure` chain rather than the caller's `ein` slot — same scoping the
old recursive `run_body(_, &pe)` path gave.

Mutual recursion still doesn't work — that's a `define` tie-back
limitation, not a TCO one. Defining `evn?` before `od?` captures an
env where `od?` is unbound; the tie-back only adds the self-binding.

## Interpreter limitations (design, not bug)

- **Arena, not GC.** Cells live in two bump-pointer arenas: `perm`
  (top-level definitions and the value graph each one pins) and
  `trans` (parse cells, intermediate evals, the form's printed
  result). `repl()` calls `arena_reset_trans()` between top-level
  forms; a Cheney-style `promote_v` deep-copies the value graph at
  every top-level `define`/`set!` boundary so no perm cell ever
  points into trans. Forwarding markers (`pin = -1` plus the new
  perm ptr stashed in `.car`/`.val`) break cycles. Detached trans
  chunks go onto a per-arena free list, so peak virtual address
  space is bounded by the largest form's working set. Builtin
  args-slice headers in eval's apply path are released with
  `os.free` per dispatch — without that, `rt_ensure`'s page-per-
  call allocation dominates the profile. test_huge peaks at ~2.6 MB
  under `--pages-as-heap=yes`, down from ~525 MB pre-arena (~200×).
  No real GC inside a single form, so a pathological one-shot like
  `(fib 25)` would still grow trans linearly until the form returns.
- **No bigints.** `i64` wraps silently on overflow. `(fact 21)`
  rolls over.
- **Float printing is 6-digit fixed-point.** The printer delegates
  to `strconv.f64tos`, which trims trailing zeros and the trailing
  '.' (`1.0` → `1`, `1.5` → `1.5`, `0.1` → `0.1`). It does not yet
  emit scientific notation or detect NaN/Inf — magnitudes ≥ 9e18
  print as `huge`. Graduate-to-Ryū requires `f64`↔`u64` bit-
  reinterpret in cgen.
- **String escapes are accepted but not translated.** `"\n"` in
  source lands as the two bytes `\\` + `n`, not a newline.
- **No `(load)` / file I/O builtins.** Programs come in via stdin.

## Latent issues — not observed, but the surface to watch

- Other i64 compound assigns in the cgen aren't audited site-by-site.
  Workaround #8 covers f64; if you add new arithmetic on integers
  inside a hot loop, scan the asm.
- The interactive REPL's `shift_buf(L.curstart)` assumes `parse_expr`
  always primes exactly one token of lookahead after returning success.
  A future parser change that skips that prime silently eats input.
- The `define` tie-back prepends a self-binding frame onto
  `bv.envp`. It assumes constructors always initialize `envp`. They
  do today.
- Cross-module type names (`lispcore.value`) don't resolve; we
  side-step by using bare names. If `ww`'s module resolver starts
  honoring the qualified form, the tests still work — but if it
  starts *rejecting* unqualified cross-module references, both
  files break.

## Build / test / profile

```
make            # build ./lisp
make test       # build + run lisp_test (72 in-process probes)
make demo       # cat each test_*.lisp through ./lisp
make clean
```

Demo files are pipe-driven: `cat test_arith.lisp | ./lisp`.

Valgrind:

```
valgrind --error-exitcode=2 ./lisp < test_huge.lisp
valgrind --tool=massif --pages-as-heap=yes ./lisp < test_huge.lisp
```

The memcheck leak counter reads "0 allocs / 0 frees" because
`rt_alloc` is `mmap`, not `malloc`. Memcheck still catches uninit
reads, bad accesses, and stack issues. For real heap-shape analysis
use massif with `--pages-as-heap=yes`.