Hojun-Cho
46edb8db4a
Six fixes across the toolchain, surfaced by lib/lisp porting work.
1. f64 compound assigns (`acc += d`, `-=`, `*=`, `/=`). Both stages
load slot → X1, OP X0 into X1, store back (ADDSD/SUBSD/MULSD/
DIVSD are reg-reg only). Previous MOVSD-overwrite dropped the
OP. Locals and top-level lets.
2. Top-level `[N]u8` arrays + `&arr[i]`. let_emit_size grows a
TY_ARRAY branch so zero-init DATAW lands; cgindex / N_INDEX
store / `&base[i]` all detect a global array base and use
LEAQ name(SB) instead of LEAQ (BP). TK_AMP no longer pre-
evaluates the operand as a value-load — `&base[i]` computes
base + i*esz directly. Unblocks Hare's static-buffer pattern:
strconv.{u64,i64,f64}tos graduate to module-level `*_buf`
arrays and return owned views.
3. Cross-module `pkg.Enum.MEMBER`. Nested N_DOT chains that
don't fold to a known shape now emit `MOVQ <leaf>(SB), AX`
(mirrors the bare-IDENT unresolved fallback), so isolation
probes — and the test 990 cgen-match floor — stay consistent
across stages. strconv exposes `base` as a real `enum i32`;
callers updated. The `main` exemption (linker entry-point
keeps bare name even when not exported) mirrors C-side
collectmods into selfhost cgendecl.
4. Sum-typed parameter ABI. lib/bytes.{index,rindex} take
`(u8 | []u8)` needle; lib/strings.byteindex / rbyteindex take
`(str | rune)` needle (Hare-shaped; the byte-wise misnomer
`index` is dropped). tagged_arg_size cap bumps to 48 (6 int
regs), with a new partial-fit branch on the callee: when an
N-word tagged arg overflows remaining regs, fill what fits and
stitch the rest from positive BP offsets. scanlocals MCASE
handles slice binds (24B) and walks each arm with a saved /
restored seenmark set so two arms naming the same local each
get their own slot — matches cstage's per-arm scope reset.
5. 4-reg tagged-return ABI (AX=tag, DX=word0, CX=word1, R8=word2),
up from 3 regs. Slice-payload variants (`([]T | E)`, slot 32B)
round-trip ptr/len/cap end-to-end. Every receive site updates:
let-init via cgwidentaggedstore, match scrutinee spill, cgindex
tagged-element load (both N_IDENT and fallback bases),
pushargsrev tagged-ident arg (reads word count from slot size),
cgreturn slice variant in the shuffle path.
6. `expr: TaggedAlias` is a widening, not a re-interpret. C cgen +
selfhost cgwidentaggedstore peel an N_CAST whose destination IS
the union — so cgexpr's natural shape (str: AX=ptr, BX=len;
slice: AX=ptr, BX=len, CX=cap) is consumed by the matching
concrete-variant branch instead of being misread as a tagged
AX/DX/CX triple. Inner casts to a concrete variant (`7: i32`)
keep their type for proper tag lookup. `[N]Alias` arrays
resolve element size via slotsize + aliaslookup, and aliaslookup
strips a `pkg.` prefix so cross-module references work.
lib/fmt grows `formattable = (i64 | str | bool | rune)` plus
`printv` / `printlnv` taking an explicit `[]formattable` slice (the
receive side of Hare's `args: formattable...`). Call-site variadic
gather isn't wired — callers either hand-build the slice or compose
strconv.i64tos + strings.concat.
700_e2e: 114 → 123 rows (f64 compound, top-level u8 arrays + `&buf[i]`,
pkg.Enum.MEMBER, sum-typed (str|rune) and (u8|[]u8) params, 4-reg
slice-return ABI, formattable array). 26/26 tests, bootstrap stable
through ww4.
177 lines
8.2 KiB
Markdown
177 lines
8.2 KiB
Markdown
examples/lisp — tiny Lisp interpreter in pure ww. Demo program; not a
|
||
production interpreter. Drives the wwstage cgen (`out/bin/w6c_ww`) by
|
||
default; the Makefile sets `WW_W6C` so `make` picks the ww-built
|
||
backend rather than the C bootstrap one.
|
||
|
||
## Layout
|
||
|
||
- `lispcore.ww` interpreter module: types, lexer, parser, env,
|
||
eval, apply, printer, REPL. Everything the entry
|
||
point and the test driver consume is `export`-ed.
|
||
- `lisp.ww` entry point; `use lispcore;` + `main()`.
|
||
- `lisp_test.ww` in-process test driver (101 probes). Built as a
|
||
standalone binary, exec'd directly — `ww test`
|
||
drops `-I` in single-file mode, so the Makefile
|
||
runs the binary itself.
|
||
- `test_*.lisp` demo source files (`cat test_X.lisp | ./lisp`).
|
||
|
||
The lispcore.ww + lisp.ww split exists so the tests can `use` the
|
||
interpreter functions. Both files combine under the same module name
|
||
(the directory's basename, `lisp`), because `ww`'s module resolver
|
||
takes the *containing-directory* basename when finding a sibling.
|
||
Cross-module type prefixes like `lispcore.value` don't resolve in the
|
||
test — use the bare names.
|
||
|
||
## wwstage cgen workarounds at play
|
||
|
||
The wwstage cgen has been brought up to par on the bug classes this
|
||
demo originally exercised. The remaining workarounds and the
|
||
historical bugs they sidestepped are listed below; "(retired)" marks
|
||
items the cgen now handles natively, kept here only as a record of
|
||
the shape so a regression is easy to recognise.
|
||
|
||
1. **(retired) Top-level `[N]T` arrays mis-address inside functions.**
|
||
`globalarr[i]` used to lower to `LEAQ (BP), BX`. cgindex/cgassign
|
||
now detect top-level array idents and emit `LEAQ name(SB), BX`,
|
||
and emitletdataw lays the array bytes into DATAW.
|
||
|
||
2. **(retired) `global_ptr[i]` mis-addresses.** cgindex/cgassign
|
||
detect top-level `*T` idents and emit `MOVQ name(SB), BX` with
|
||
the correct element scaling. `lispcore`'s symbol interner is back
|
||
to plain `sym_blob[off + i]` style; no `let blob: *u8 = sym_blob`
|
||
aliasing.
|
||
|
||
3. **(retired) Two-level field write through a non-pointer
|
||
sub-struct.** cgassign/cgdot now handle the chained
|
||
`(*L).cur.kind` shape both as read and write. The lexer keeps the
|
||
flattened `curkind` / `curival` / … fields for now because every
|
||
call site uses them; un-flattening is a stylistic improvement, not
|
||
a correctness fix.
|
||
|
||
4. **(partially retired) f64 through every boundary.** Struct-field
|
||
`p.fval = v` through `*T`, `*p = v` for `*f64`, and `alloc(T{…})`
|
||
sugar for f64 fields all route through X0 now (`vfloat`,
|
||
`promote_v`, `to_f64` are back to direct `p.fval = …` /
|
||
`*out = v.fval` form). Function-arg passing of an f64 struct-field
|
||
value also works (the cgen's `exprfloatkind` now recognises
|
||
`p.field` whose declared type is f64/f32).
|
||
|
||
5. **(retired) `alloc(value{ text = s })` writes only `s.ptr`.** Both
|
||
the `alloc(T{…})` builtin and the bare struct-literal init now
|
||
emit both halves of the str. `vstr` still does manual
|
||
`p.text = s` (semantically equivalent, no longer required).
|
||
|
||
6. **(retired) `(slice | E)` tagged-union returns drop `slice.len`.**
|
||
The return ABI now uses 4 regs (AX=tag, DX=ptr, CX=len, R8=cap),
|
||
covering slice-payload variants up to 32B. The inline args-eval
|
||
in `eval` is still there but no longer required — `apply` could
|
||
factor it back out into a `(slice | rterror)` helper. Left as a
|
||
readability cleanup; not a correctness fix.
|
||
|
||
7. **(retired) `xs[i].kind` drops the trailing field load.** cgdot
|
||
now handles N_INDEX bases. Builtins are back to
|
||
`xs[0].kind` / `xs[0].car` directly — no `let p = xs[0];` first.
|
||
|
||
8. **f64 compound assigns are mis-lowered to `acc = d` (no OP).**
|
||
Still present. Write the explicit form: `acc = acc + f`. Integer
|
||
compound assigns work fine.
|
||
|
||
9. **(retired in practice) `let r = call(); foreign_call(); return
|
||
r?;` corrupts `r`.** The wwstage cgen now spills the AX/DX/CX
|
||
triple to the local's 24-byte slot at the assignment point,
|
||
matching what cstage emits — the foreign call in between no longer
|
||
clobbers an unspilled half. We still pre-match the union inline in
|
||
eval's BUILTIN apply path so the `os.free` happens *after*
|
||
classification rather than after the unwrap.
|
||
|
||
If a new function "should work but acts weird", check #6 first.
|
||
|
||
## Tail-call optimization
|
||
|
||
`eval` is a single `for(true)` trampoline; a tail position rewrites
|
||
`v` (current expr) and `cure` (current env) in place and `continue`s
|
||
instead of recursing. Tail positions are:
|
||
|
||
- the chosen branch of `if`
|
||
- the last form of `begin` / `let` / a lambda body
|
||
- a direct call in any of the above
|
||
|
||
A `tailed` flag flips on the first jump into a fresh scope (LET
|
||
binding, lambda body) so subsequent `define`/`set!` mutate the local
|
||
`cure` chain rather than the caller's `ein` slot — same scoping the
|
||
old recursive `run_body(_, &pe)` path gave.
|
||
|
||
Mutual recursion still doesn't work — that's a `define` tie-back
|
||
limitation, not a TCO one. Defining `evn?` before `od?` captures an
|
||
env where `od?` is unbound; the tie-back only adds the self-binding.
|
||
|
||
## Interpreter limitations (design, not bug)
|
||
|
||
- **Arena, not GC.** Cells live in two bump-pointer arenas: `perm`
|
||
(top-level definitions and the value graph each one pins) and
|
||
`trans` (parse cells, intermediate evals, the form's printed
|
||
result). `repl()` calls `arena_reset_trans()` between top-level
|
||
forms; a Cheney-style `promote_v` deep-copies the value graph at
|
||
every top-level `define`/`set!` boundary so no perm cell ever
|
||
points into trans. Forwarding markers (`pin = -1` plus the new
|
||
perm ptr stashed in `.car`/`.val`) break cycles. Detached trans
|
||
chunks go onto a per-arena free list, so peak virtual address
|
||
space is bounded by the largest form's working set. Builtin
|
||
args-slice headers in eval's apply path are released with
|
||
`os.free` per dispatch — without that, `rt_ensure`'s page-per-
|
||
call allocation dominates the profile. test_huge peaks at ~2.6 MB
|
||
under `--pages-as-heap=yes`, down from ~525 MB pre-arena (~200×).
|
||
No real GC inside a single form, so a pathological one-shot like
|
||
`(fib 25)` would still grow trans linearly until the form returns.
|
||
- **No bigints.** `i64` wraps silently on overflow. `(fact 21)`
|
||
rolls over.
|
||
- **Float printing is 6-digit fixed-point.** The printer delegates
|
||
to `strconv.f64tos`, which trims trailing zeros and the trailing
|
||
'.' (`1.0` → `1`, `1.5` → `1.5`, `0.1` → `0.1`). It does not yet
|
||
emit scientific notation or detect NaN/Inf — magnitudes ≥ 9e18
|
||
print as `huge`. Graduate-to-Ryū requires `f64`↔`u64` bit-
|
||
reinterpret in cgen.
|
||
- **String escapes are accepted but not translated.** `"\n"` in
|
||
source lands as the two bytes `\\` + `n`, not a newline.
|
||
- **No `(load)` / file I/O builtins.** Programs come in via stdin.
|
||
|
||
## Latent issues — not observed, but the surface to watch
|
||
|
||
- Other i64 compound assigns in the cgen aren't audited site-by-site.
|
||
Workaround #8 covers f64; if you add new arithmetic on integers
|
||
inside a hot loop, scan the asm.
|
||
- The interactive REPL's `shift_buf(L.curstart)` assumes `parse_expr`
|
||
always primes exactly one token of lookahead after returning success.
|
||
A future parser change that skips that prime silently eats input.
|
||
- The `define` tie-back prepends a self-binding frame onto
|
||
`bv.envp`. It assumes constructors always initialize `envp`. They
|
||
do today.
|
||
- Cross-module type names (`lispcore.value`) don't resolve; we
|
||
side-step by using bare names. If `ww`'s module resolver starts
|
||
honoring the qualified form, the tests still work — but if it
|
||
starts *rejecting* unqualified cross-module references, both
|
||
files break.
|
||
|
||
## Build / test / profile
|
||
|
||
```
|
||
make # build ./lisp
|
||
make test # build + run lisp_test (72 in-process probes)
|
||
make demo # cat each test_*.lisp through ./lisp
|
||
make clean
|
||
```
|
||
|
||
Demo files are pipe-driven: `cat test_arith.lisp | ./lisp`.
|
||
|
||
Valgrind:
|
||
|
||
```
|
||
valgrind --error-exitcode=2 ./lisp < test_huge.lisp
|
||
valgrind --tool=massif --pages-as-heap=yes ./lisp < test_huge.lisp
|
||
```
|
||
|
||
The memcheck leak counter reads "0 allocs / 0 frees" because
|
||
`rt_alloc` is `mmap`, not `malloc`. Memcheck still catches uninit
|
||
reads, bad accesses, and stack issues. For real heap-shape analysis
|
||
use massif with `--pages-as-heap=yes`.
|