Hojun-Cho/ww
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

lib/encoding/utf8+test: Hare port (decoder / next / encoderune / runesz / utf8sz / validate)

Browse Source 
Hoehrmann DFA from ref/hare/encoding/utf8/decodetable.ha flattened
to 1D [2048]i8 (task #20: 2D-array jagged cgen still pending);
encoderune takes a caller buffer matching lib/encoding/hex.encode;
done/more/invalid all spelled as plain void aliases per lib/io's
eof precedent. Surface ports decoder + decode + next + encoderune
+ runesz + utf8sz + validate from ref/hare/encoding/utf8/{types,
decode,encode,rune}.ha. next() polarity rewritten from Hare's
`(state-1):uint >> 31` to an explicit `if state == 0` branch
because ww's uint is 64-bit (cmd/wcc/type.c:58); same effect, no
hidden 32-bit assumption.

Deferred (no in-tree callers): prev, slice, position, remaining,
appendrune, strencode, strdecode. String iteration (chars/
newchars/nextchar in the session-4 draft) dropped per Hare
discipline — belongs in lib/strings::iterator, not encoding/utf8.

Tests:
  - 968_utf8_run drives lib/encoding/utf8/utf8test.ww via ww run.
    21 @test fns: boundaries (ASCII, 2-byte, 3-byte, 4-byte
    encode/decode), surrogate/overlong/out-of-range/bad-continuation
    reject, max-in-range (U+10FFFF) accept, truncated→more, done@EOI,
    validate empty/mixed/malformed, encode/decode roundtrip. Two
    rows ported from ref/hare/encoding/utf8/decode.ha @test that
    were missing in the session-4 draft: bad-continuation
    [0xC2,0xFF]→invalid and max-in-range [0xF4,0x8F,0xBF,0xBF]→
    U+10FFFF.
  - 9xx stdlib runtime slot range extended from 970-989 to 960-989
    to accommodate utf8 at 968 (970-989 block was full).

90/90 ok. 995_self_rebuild stays green (ww2==ww3==ww4 byte-id).
This commit is contained in:
Hojun-Cho
2026-05-17 23:46:34 +09:00
parent 7e0c280691
commit 793734c1e0
4 changed files with 743 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
Makefile
View File

@@ -260,7 +260,7 @@ TESTS = $(BIN)/test_smoke $(BIN)/test_lex $(BIN)/test_parse $(BIN)/test_check \
$(BIN)/test_shlex_run $(BIN)/test_getenv_run $(BIN)/test_dirs_run \
$(BIN)/test_stat_run $(BIN)/test_time_run \
$(BIN)/test_intdiv_signed \
$(BIN)/test_hex_run \
$(BIN)/test_hex_run $(BIN)/test_utf8_run \
$(BIN)/test_memio_run $(BIN)/test_temp_run $(BIN)/test_getopt_run \
$(BIN)/test_base32_run $(BIN)/test_base64_run \
$(BIN)/test_adler32_run $(BIN)/test_crc16_run \
@@ -634,6 +634,10 @@ $(BIN)/test_hex_run: test/wcc/979_hex_run.c $(BIN)/ww $(BIN)/w6c \
$(BIN)/w6a $(BIN)/w6l $(LIB)/libwwrt.a | $(BIN)
$(CC) $(CFLAGS) -o $@ $<
$(BIN)/test_utf8_run: test/wcc/968_utf8_run.c $(BIN)/ww $(BIN)/w6c \
$(BIN)/w6a $(BIN)/w6l $(LIB)/libwwrt.a | $(BIN)
$(CC) $(CFLAGS) -o $@ $<
$(BIN)/test_memio_run: test/wcc/980_memio_run.c $(BIN)/ww $(BIN)/w6c \
$(BIN)/w6a $(BIN)/w6l $(LIB)/libwwrt.a | $(BIN)
$(CC) $(CFLAGS) -o $@ $<