Files
ww/lib
Hojun-Cho 9526d21007 lib/encoding/utf8+test: Hare port (prev + slice + position + remaining)
Port four deferred utf8 functions per c2 backlog (utf8.ww:18
pre-port). All are non-trivial enough that the test rows mirror
Hare's @test fn decode/slice bodies (ref/hare/encoding/utf8/
decode.ha:85-198) row-for-row.

  - prev       ref/hare/encoding/utf8/decode.ha:52-71
  - remaining  ref/hare/encoding/utf8/decode.ha:74
  - slice      ref/hare/encoding/utf8/decode.ha:80-83
  - position   ref/hare/encoding/utf8/decode.ha:203

prev walks back from d.offs to a byte that could start a codepoint
(state-0 dfa cell != -1), re-decodes forward from there, and
confirms the forward decode lands back at the original offset.
Returns done at start-of-input; invalid if no initial byte appears
within 4 steps (no legal UTF-8 codepoint exceeds 4 bytes) or if
the forward decode shortcircuits to more/invalid or lands at a
different offset than expected.

Two structural deltas from the Hare source:

  - Hare's `for (d.offs < len(d.src); d.offs -= 1)` relies on
    size_t wrap-around to exit when offs underflows past 0; ww's
    offs is i32 (utf8.ww:204), so the same exit is spelled
    `d.offs >= 0`.

  - Hare's `defer d.offs = t` restores offs after the return; ww
    has no defer, so the restore is inlined in each match arm.

slice asserts the Hare precondition (same source + begin.offs <=
end.offs) via abort; Hare uses assert(). position is a one-liner
returning d.offs (Hare uses size, ww uses i32 per lib/CLAUDE.md
"indices use the underlying length type").

The `bi: i32 = b: i32;` indirection in prev's dfa lookup is
required because `dfa[b: i32]` parses as a slice expression
`dfa[b : i32]` where `i32` becomes the upper bound. The let-binding
matches the existing pattern at utf8.ww:236-238 in the c1 next()
port.

Tests: 14 new @test fns in utf8test.ww (signalled 22-35):
prev_done_at_start, prev_one/two/three/four_byte (round-trip
forward+reverse), prev_mixed_roundtrip (full forward then full
reverse on the same こんにちは+NUL input Hare uses at
decode.ha:85-111), prev_continuation_only_more (Hare's
[0xA0,0xA1] more case at decode.ha:117), prev_incomplete /
surrogate / overlong / extracont_invalid (decode.ha:120-150),
prev_max_in_range (decode.ha:158-163; pins state-7 acceptance via
reverse decode), prev_min_out_of_range (decode.ha:166-169; the
only case that trips prev's 4-step-bound arm), and
remaining_slice_position mirroring decode.ha:172-198.

The Hare slice @test idiom `let d2 = d1` (struct copy) miscompiles
in both stages (cstage + wwstage zero-init the rhs instead of
copying — task #32, Class A but bootstrap-byte-id-symmetric, so
995 doesn't catch it). The ww test uses two parallel
`decode(src[0:16])` calls to produce two decoders with the same
src.ptr; coverage on slice() is equivalent (exercises the same
source-different-offs pattern). Divergence cited inline.

117/117 ok. 995_self_rebuild stays green (ww2==ww3==ww4 byte-id).
2026-05-18 22:00:15 +09:00
..