lib/strings+test: Hare port (iterator + next)

Forward UTF-8 rune cursor per ref/hare/strings/iter.ha; iterator
flattens Hare's anon-embedded utf8::decoder to explicit
offs/src/reverse fields, next() copy-in/copy-out a local decoder
and aborts on more/invalid per move()'s discipline.

The iterator flattens Hare's anonymous-embedded utf8::decoder
(ref/hare/strings/iter.ha:6-9) to explicit offs/src/reverse
fields because ww has no anon-embed syntax. reverse is retained
on the struct so riter populates it once utf8.prev (reverse DFA)
and strings.prev land.

next() copies the iterator's offs/src into a local utf8.decoder,
delegates to utf8.next, then writes offs back; copy-in/copy-out
is the cost of the flattened layout. more/invalid arms abort with
"strings.next: invalid UTF-8", mirroring Hare's move()
(ref/hare/strings/iter.ha:51-58) which aborts unconditionally
on those arms.

Deferred surface (no in-tree caller; follow-up tasks): prev, riter,
iterstr, slice, position, move. prev specifically needs utf8.prev
(reverse DFA), which isn't on the lib/encoding/utf8 surface yet.

lib/strings/strings.ww moves off test/wcc/900_stdlib.c's
standalone-compile list per the existing bufio/fmt/os precedent:
the iterator's (rune | utf8.done) return type and the local
utf8.decoder reference need cross-module type resolution, which
the standalone w6c path doesn't do. Runtime coverage stays at
966_strings_run, which now exercises 21 signalled cases (was 15).

The iterator's Hare-faithful `reverse: bool` field surfaced #33
(cstage cgen narrow-sret-field copy mis-width) during this port's
pre-flight; that fix landed at 0d96196 ahead of this commit.
Future bisecters tracking a narrow-field-related regression in
lib/strings or sret-aware stdlib growth should consult #33 + this
commit's bracket. The 4-arm match on utf8.next return surfaced
#31 (wwstage fnretlookupmod) earlier in the same chain (b787641).

Tests:
  - 6 new @test fns in stringstest.ww (signalled 16-21): empty,
    ASCII, 2-byte (café), 3-byte (こんにちは), 4-byte (🦀rust),
    mixed-width ("Hello, 世界! 🌍"). Each verifies forward
    iteration, done@EOI, and repeated next-after-done stays done
    (iter_empty). Multibyte literal limitation handled via
    `0xE9u32: rune` cast per existing pattern at stringstest.ww:82.

104/104 ok. 995_self_rebuild stays green (ww2==ww3==ww4 byte-id).
This commit is contained in:
2026-05-18 12:16:22 +09:00
parent 0d96196f90
commit d09197af8e
6 changed files with 330 additions and 10 deletions

View File

@@ -22,6 +22,13 @@
// `os.alloc` aborts on OOM (no `nomem` type), so we return plain
// `str`. Empty input returns `{nil, 0}`; Hare returns the static
// empty string — same observable result.
// - `iterator` is flattened (`offs`, `src`, `reverse` fields).
// Hare uses anonymous-embedded `utf8::decoder`
// (ref/hare/strings/iter.ha:6-9); ww has no anonymous-embed
// syntax. `next` copies the iterator's `offs`/`src` into a local
// `utf8.decoder` for the call, then writes `offs` back. `prev` /
// `riter` / `iterstr` / `slice` / `position` are deferred — no
// in-tree caller; `prev` needs `utf8.prev` (reverse DFA).
use bytes;
use utf8;
@@ -268,3 +275,43 @@ export fn rtrim(input: str, exclude: rune) str = {
export fn trim(input: str, exclude: rune) str = {
return ltrim(rtrim(input, exclude), exclude);
};
// iterator — forward UTF-8 rune cursor over a `str`. Layout flattens
// Hare's anonymous-embedded `utf8::decoder`
// (ref/hare/strings/iter.ha:6-9) to explicit fields; `reverse` is
// retained on the type because `riter` will populate it once `prev` /
// `utf8.prev` land. May be copied to save state.
export type iterator = struct {
offs: i32,
src: []u8,
reverse: bool,
};
// iter — initialize a forward iterator at the start of `src`.
// ref/hare/strings/iter.ha:24.
export fn iter(src: str) iterator = {
let r: iterator;
r.src = toutf8(src);
r.offs = 0;
r.reverse = false;
return r;
};
// next — advance the iterator one rune. Returns `utf8.done` at end
// of input. Aborts on `more` / `invalid` — mirrors Hare's
// ref/hare/strings/iter.ha:51-58 `move()`, which aborts unconditionally
// on those arms ("Invalid UTF-8 string (this should not happen)").
//
// Copy-in / copy-out is the cost of flattening the embedded decoder;
// see the iterator divergence note at the top of the file.
export fn next(it: *iterator) (rune | utf8.done) = {
let d: utf8.decoder;
d.src = it.src;
d.offs = it.offs;
match (utf8.next(&d)) {
case let r: rune => { it.offs = d.offs; return r; };
case let dn: utf8.done => return dn;
case let m: utf8.more => abort("strings.next: invalid UTF-8");
case let e: utf8.invalid => abort("strings.next: invalid UTF-8");
};
};