Hojun-Cho
d9aba892f6
Makefile sets WW_W6C=$(BIN)/w6c_ww so `make`, `make test`, and `make demo` all use the ww-built backend. With the wwstage cgen fixes in selfhost/cmd/wcc/ the demo no longer needs to dodge: - bug #1+#2 (global addressing): symbol interner indexes sym_off / sym_len / sym_blob directly. No `let blob = sym_blob;` aliasing. - bug #3 (chained non-pointer sub-struct field): not retired here (lexer.cur is still flattened) but the cgen now handles the shape; un-flattening is cosmetic. - bug #4 (f64 through every boundary): vfloat writes p.fval = v directly; promote_v's FLOAT branch is one assign; to_f64 reads v.ival as f64 / v.fval directly. Drops fbuf, FVAL_OFF, copybytes. - bug #7 (xs[i].field): builtins write xs[0].kind / xs[0].car directly; no `let p = xs[0];` first. lisp_test still 101/101. CLAUDE.md marks each historical bug as retired or still load- bearing; #6 (slice-len in tagged-union return) and #8 (f64 compound assign) are the remaining shapes to avoid.
176 lines
8.1 KiB
Markdown
176 lines
8.1 KiB
Markdown
examples/lisp — tiny Lisp interpreter in pure ww. Demo program; not a
|
||
production interpreter. Drives the wwstage cgen (`out/bin/w6c_ww`) by
|
||
default; the Makefile sets `WW_W6C` so `make` picks the ww-built
|
||
backend rather than the C bootstrap one.
|
||
|
||
## Layout
|
||
|
||
- `lispcore.ww` interpreter module: types, lexer, parser, env,
|
||
eval, apply, printer, REPL. Everything the entry
|
||
point and the test driver consume is `export`-ed.
|
||
- `lisp.ww` entry point; `use lispcore;` + `main()`.
|
||
- `lisp_test.ww` in-process test driver (101 probes). Built as a
|
||
standalone binary, exec'd directly — `ww test`
|
||
drops `-I` in single-file mode, so the Makefile
|
||
runs the binary itself.
|
||
- `test_*.lisp` demo source files (`cat test_X.lisp | ./lisp`).
|
||
|
||
The lispcore.ww + lisp.ww split exists so the tests can `use` the
|
||
interpreter functions. Both files combine under the same module name
|
||
(the directory's basename, `lisp`), because `ww`'s module resolver
|
||
takes the *containing-directory* basename when finding a sibling.
|
||
Cross-module type prefixes like `lispcore.value` don't resolve in the
|
||
test — use the bare names.
|
||
|
||
## wwstage cgen workarounds at play
|
||
|
||
The wwstage cgen has been brought up to par on the bug classes this
|
||
demo originally exercised. The remaining workarounds and the
|
||
historical bugs they sidestepped are listed below; "(retired)" marks
|
||
items the cgen now handles natively, kept here only as a record of
|
||
the shape so a regression is easy to recognise.
|
||
|
||
1. **(retired) Top-level `[N]T` arrays mis-address inside functions.**
|
||
`globalarr[i]` used to lower to `LEAQ (BP), BX`. cgindex/cgassign
|
||
now detect top-level array idents and emit `LEAQ name(SB), BX`,
|
||
and emitletdataw lays the array bytes into DATAW.
|
||
|
||
2. **(retired) `global_ptr[i]` mis-addresses.** cgindex/cgassign
|
||
detect top-level `*T` idents and emit `MOVQ name(SB), BX` with
|
||
the correct element scaling. `lispcore`'s symbol interner is back
|
||
to plain `sym_blob[off + i]` style; no `let blob: *u8 = sym_blob`
|
||
aliasing.
|
||
|
||
3. **(retired) Two-level field write through a non-pointer
|
||
sub-struct.** cgassign/cgdot now handle the chained
|
||
`(*L).cur.kind` shape both as read and write. The lexer keeps the
|
||
flattened `curkind` / `curival` / … fields for now because every
|
||
call site uses them; un-flattening is a stylistic improvement, not
|
||
a correctness fix.
|
||
|
||
4. **(partially retired) f64 through every boundary.** Struct-field
|
||
`p.fval = v` through `*T`, `*p = v` for `*f64`, and `alloc(T{…})`
|
||
sugar for f64 fields all route through X0 now (`vfloat`,
|
||
`promote_v`, `to_f64` are back to direct `p.fval = …` /
|
||
`*out = v.fval` form). Function-arg passing of an f64 struct-field
|
||
value also works (the cgen's `exprfloatkind` now recognises
|
||
`p.field` whose declared type is f64/f32).
|
||
|
||
5. **(retired) `alloc(value{ text = s })` writes only `s.ptr`.** Both
|
||
the `alloc(T{…})` builtin and the bare struct-literal init now
|
||
emit both halves of the str. `vstr` still does manual
|
||
`p.text = s` (semantically equivalent, no longer required).
|
||
|
||
6. **`(slice | E)` tagged-union returns drop `slice.len`.** Still
|
||
present. The wwstage return convention is AX=tag, DX=payload1,
|
||
CX=payload2 — a 24-byte slice header doesn't fit. Inline the
|
||
slice-building loop into the caller. See `eval`'s argument-eval
|
||
inline.
|
||
|
||
7. **(retired) `xs[i].kind` drops the trailing field load.** cgdot
|
||
now handles N_INDEX bases. Builtins are back to
|
||
`xs[0].kind` / `xs[0].car` directly — no `let p = xs[0];` first.
|
||
|
||
8. **f64 compound assigns are mis-lowered to `acc = d` (no OP).**
|
||
Still present. Write the explicit form: `acc = acc + f`. Integer
|
||
compound assigns work fine.
|
||
|
||
9. **(retired in practice) `let r = call(); foreign_call(); return
|
||
r?;` corrupts `r`.** The wwstage cgen now spills the AX/DX/CX
|
||
triple to the local's 24-byte slot at the assignment point,
|
||
matching what cstage emits — the foreign call in between no longer
|
||
clobbers an unspilled half. We still pre-match the union inline in
|
||
eval's BUILTIN apply path so the `os.free` happens *after*
|
||
classification rather than after the unwrap.
|
||
|
||
If a new function "should work but acts weird", check #6 first.
|
||
|
||
## Tail-call optimization
|
||
|
||
`eval` is a single `for(true)` trampoline; a tail position rewrites
|
||
`v` (current expr) and `cure` (current env) in place and `continue`s
|
||
instead of recursing. Tail positions are:
|
||
|
||
- the chosen branch of `if`
|
||
- the last form of `begin` / `let` / a lambda body
|
||
- a direct call in any of the above
|
||
|
||
A `tailed` flag flips on the first jump into a fresh scope (LET
|
||
binding, lambda body) so subsequent `define`/`set!` mutate the local
|
||
`cure` chain rather than the caller's `ein` slot — same scoping the
|
||
old recursive `run_body(_, &pe)` path gave.
|
||
|
||
Mutual recursion still doesn't work — that's a `define` tie-back
|
||
limitation, not a TCO one. Defining `evn?` before `od?` captures an
|
||
env where `od?` is unbound; the tie-back only adds the self-binding.
|
||
|
||
## Interpreter limitations (design, not bug)
|
||
|
||
- **Arena, not GC.** Cells live in two bump-pointer arenas: `perm`
|
||
(top-level definitions and the value graph each one pins) and
|
||
`trans` (parse cells, intermediate evals, the form's printed
|
||
result). `repl()` calls `arena_reset_trans()` between top-level
|
||
forms; a Cheney-style `promote_v` deep-copies the value graph at
|
||
every top-level `define`/`set!` boundary so no perm cell ever
|
||
points into trans. Forwarding markers (`pin = -1` plus the new
|
||
perm ptr stashed in `.car`/`.val`) break cycles. Detached trans
|
||
chunks go onto a per-arena free list, so peak virtual address
|
||
space is bounded by the largest form's working set. Builtin
|
||
args-slice headers in eval's apply path are released with
|
||
`os.free` per dispatch — without that, `rt_ensure`'s page-per-
|
||
call allocation dominates the profile. test_huge peaks at ~2.6 MB
|
||
under `--pages-as-heap=yes`, down from ~525 MB pre-arena (~200×).
|
||
No real GC inside a single form, so a pathological one-shot like
|
||
`(fib 25)` would still grow trans linearly until the form returns.
|
||
- **No bigints.** `i64` wraps silently on overflow. `(fact 21)`
|
||
rolls over.
|
||
- **Float printing is 6-digit fixed-point.** The printer delegates
|
||
to `strconv.f64tos`, which trims trailing zeros and the trailing
|
||
'.' (`1.0` → `1`, `1.5` → `1.5`, `0.1` → `0.1`). It does not yet
|
||
emit scientific notation or detect NaN/Inf — magnitudes ≥ 9e18
|
||
print as `huge`. Graduate-to-Ryū requires `f64`↔`u64` bit-
|
||
reinterpret in cgen.
|
||
- **String escapes are accepted but not translated.** `"\n"` in
|
||
source lands as the two bytes `\\` + `n`, not a newline.
|
||
- **No `(load)` / file I/O builtins.** Programs come in via stdin.
|
||
|
||
## Latent issues — not observed, but the surface to watch
|
||
|
||
- Other i64 compound assigns in the cgen aren't audited site-by-site.
|
||
Workaround #8 covers f64; if you add new arithmetic on integers
|
||
inside a hot loop, scan the asm.
|
||
- The interactive REPL's `shift_buf(L.curstart)` assumes `parse_expr`
|
||
always primes exactly one token of lookahead after returning success.
|
||
A future parser change that skips that prime silently eats input.
|
||
- The `define` tie-back prepends a self-binding frame onto
|
||
`bv.envp`. It assumes constructors always initialize `envp`. They
|
||
do today.
|
||
- Cross-module type names (`lispcore.value`) don't resolve; we
|
||
side-step by using bare names. If `ww`'s module resolver starts
|
||
honoring the qualified form, the tests still work — but if it
|
||
starts *rejecting* unqualified cross-module references, both
|
||
files break.
|
||
|
||
## Build / test / profile
|
||
|
||
```
|
||
make # build ./lisp
|
||
make test # build + run lisp_test (72 in-process probes)
|
||
make demo # cat each test_*.lisp through ./lisp
|
||
make clean
|
||
```
|
||
|
||
Demo files are pipe-driven: `cat test_arith.lisp | ./lisp`.
|
||
|
||
Valgrind:
|
||
|
||
```
|
||
valgrind --error-exitcode=2 ./lisp < test_huge.lisp
|
||
valgrind --tool=massif --pages-as-heap=yes ./lisp < test_huge.lisp
|
||
```
|
||
|
||
The memcheck leak counter reads "0 allocs / 0 frees" because
|
||
`rt_alloc` is `mmap`, not `malloc`. Memcheck still catches uninit
|
||
reads, bad accesses, and stack issues. For real heap-shape analysis
|
||
use massif with `--pages-as-heap=yes`.
|