Hojun-Cho/ww
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

lib: collapse the parallel vstream scaffold onto the single Hare io surface (#94 fold-eFinal)

Browse Source 
The Option-C parallel _v vstream API was scaffolding to bring the io stack up alongside the old surface; carrying both permanently is a rule-9 divergence from ref/hare, which has exactly one io surface. Collapse onto that surface (stream = *vtable, ref/hare/io/stream.ha) and rename the _v symbols to their Hare names (io vstream->stream, fmt vfprint->fprint, bufio/memio/log surfaces, log.new). Deletes the 4 lib/*/vstream.ww scaffold files; regenerates w6c/wwdump combined.ww. cstage and wwstage stay byte-identical and combined_ww_fresh holds; all 220 tests pass.
This commit is contained in:
Hojun-Cho
2026-05-30 03:24:23 +09:00
parent 80184a3acf
commit 7d39f6d623
31 changed files with 2371 additions and 4834 deletions
Download Patch File Download Diff File Expand all files Collapse all files

579
lib/bufio/bufio.ww
View File

@@ -1,84 +1,49 @@
// bufio — buffered I/O over [[io.stream]]. Subset of Hare's bufio::
// surface (ref/hare/bufio/{scanner,stream}.ha).
// bufio — buffered I/O over [[io.stream]] (= `*io.vtable`). Subset of
// Hare's bufio:: surface (ref/hare/bufio/{scanner,stream}.ha). Project
// #94 fold-eFinal.
//
// Surface today:
// Surface:
//
// bufio.newscanner(s: *scanner, src: *io.stream, buf: []u8) void
// bufio.finish (s: *scanner) void
// bufio.scanbyte (s: *scanner) (u8 | io.eof | io.closed)
// bufio.scantok (s: *scanner, delim: u8)
// ([]u8 | io.eof | io.closed | overflow)
// bufio.scanline (s: *scanner) (str | io.eof | io.closed | overflow)
// bufio.newscannerbuf(src: io.stream, buf: []u8) scanner
// bufio.finish (s: *scanner) void
// bufio.scanbyte (s: *scanner) (u8 | io.eof | io.error)
// bufio.scanbytes (s: *scanner, delim: u8)
// ([]u8 | io.eof | io.error | overflow)
// bufio.scanline (s: *scanner) (str | io.eof | io.error | overflow)
//
// bufio.init (b: *stream, src: *io.stream, rbuf: []u8, wbuf: []u8) void
// bufio.flush (b: *stream) (void | io.closed)
// bufio.init (src: io.stream, rbuf: []u8, wbuf: []u8) stream
// bufio.flush (b: *stream) (void | io.error)
// bufio.setflush (b: *stream, bs: []u8) void
// bufio.unread (b: *stream, buf: []u8) void
// bufio.isbuffered (s: *io.stream) bool
// bufio.isbuffered (s: io.stream) bool
//
// Divergence from Hare:
// VALUE-RETURN (Hare ref/hare/bufio/stream.ha:69 init, scanner.ha:92
// newscanner_buf): each constructor builds in a local `let r: T;`,
// field-assigns every slot, and `return r;`. The caller owns the
// returned struct (stack ownership, no-GC). For the buffered stream
// the caller passes `&b.vt` to the io dispatchers; the scanner is a
// plain read-ahead tokenizer (no embedded vtable) driven directly via
// scanbyte / scanbytes / scanline.
//
// • Hare returns the scanner / stream by value; ww cgen can't
// return structs wider than 16B by value yet, so `newscanner`
// and `init` are out-parameter shaped. Same workaround
// memio.fixed uses.
// SCOPE: this is the surface ww ships, NOT a port of Hare's full
// scanner. The features ww never implemented — auto-grow newscanner,
// multibyte-delim scanbytes / scanstring, scanrune / readbyte / readtok
// / readline / readrune / unreadrune — are a separate future feature
// fold (#217). The ported set: the fixed-buffer newscannerbuf,
// single-byte scanbytes, single-byte scanline, scanbyte, finish; and
// the buffered-stream init / setflush / flush / unread / isbuffered.
//
// • The scanner's read buffer is held flat as `(ptr, cap)` rather
// than a `[]u8` field; the scanner predates struct-held slice
// support and is wired through scanbyte/scantok/scanline. The
// new stream writer half uses slice-typed `rbuf` / `wbuf`
// fields directly — chained dot through a struct slice field
// works after task #6.
// Cast workaround per #206-payoff (ken: KEEP the explicit casts; they
// are cgen-neutral and sidestep the #214 over-acceptance surface).
// `(&fn_name): *io.<role>` at each vtable store + the isbuffered
// fn-ptr-equality comparand — the #206 cast-drop is gated on #214.
//
// • Hare's `bufio::stream` discriminates r-only / w-only / r+w
// via three different vtable singletons (vtable_r, vtable_w,
// vtable_rw); ww's [[io.stream]] vtable always carries all
// three callbacks, so stream always installs `bread` /
// `bwrite` / `bclose`. A zero-length rbuf or wbuf makes the
// matching callback degenerate (rbuf=[]: bread short-circuits
// to io.eof; wbuf=[]: bwrite passes through to src).
//
// • Hare's `init` also takes a `flag` argument carrying
// MANAGED_HANDLE / MANAGED_RDBUF / MANAGED_WRBUF ownership
// bits; stream never owns its buffers or src, so the
// `flags` field and the init-time argument are dropped
// wholesale. The Hare `flush: []u8` byte-set is preserved —
// [[init]] seeds it to `['\n']` (line buffering, matching
// Hare's `flag::NONE` default at stream.ha:75) and
// [[setflush]] swaps it.
//
// • Hare's `newscanner` allocates and grows the buffer up to
// `maxread`; we ship only the caller-supplied shape (Hare's
// `newscanner_buf`) and pick up the auto-grow variant when an
// append over a struct-held slice works.
//
// • EOF-handling defaults to Hare's `EOF_DISCARD`: bytes between
// the last delimiter and EOF are dropped and io.eof is returned
// on the call that would have read them. Hare's EOF_GREEDY mode
// isn't shipped (no caller needs it yet).
//
// Owning model: caller owns the scanner state, the stream state,
// the byte buffers, and the source stream. `finish` doesn't free
// the buffer and doesn't close src; it's a no-op today but stays
// on the surface so callers don't churn when bufio grows internal
// allocations. The stream vtable `close` callback flushes pending
// writes and forwards close to src; it does not free rbuf / wbuf.
//
// let mem: memio.state;
// let m: io.stream;
// memio.fixed(&mem, &m, raw[0:N]);
// let buf: [128]u8;
// let sc: bufio.scanner;
// bufio.newscanner(&sc, &m, buf[0:128]);
// match (bufio.scanline(&sc)) { ... };
// bufio.finish(&sc);
//
// let b: bufio.stream;
// let rbuf: [256]u8;
// let wbuf: [128]u8;
// bufio.init(&b, src, rbuf[0:256], wbuf[0:128]);
// let p: *io.stream = &b.vtable; // first-field embed
// io.write(p, msg);
// bufio.flush(&b);
// Mode discrimination (drew-deferred): Hare uses three vtable
// singletons (vtable_r / vtable_w / vtable_rw); ww's vtable always
// carries all three callbacks (a zero-length rbuf/wbuf degenerates the
// matching callback in-cb). Defer-handle (MANAGED_* ownership bits)
// also deferred — caller owns rbuf/wbuf/src; bclose flushes + forwards
// close but frees nothing. Both graduate with io fold-2 (#5).
package bufio;
@@ -101,157 +66,20 @@ let flushdefault: [1]u8 = [10u8];
// use of errors::overflow for the same condition.
export type overflow = !void;
export type scanner = struct {
src: *io.stream,
ptr: *u8,
cap: i32,
start: i32, // index where the pending region starts in ptr
avail: i32, // pending byte count; pending = ptr[start..start+avail]
};
// ---- buffered stream ------------------------------------------------------
// newscanner — wire `s` to read through `src` using `buf` as the
// read-ahead window. Hare returns the scanner by value
// (newscanner_buf); we take an out-parameter pending cgen support
// for wide-struct return.
export fn newscanner(s: *scanner, src: *io.stream, buf: []u8) void = {
s.src = src;
s.ptr = buf.ptr;
s.cap = buf.len;
s.start = 0;
s.avail = 0;
};
// finish — release scanner-owned resources. No-op today (buffer is
// caller-owned, src isn't closed); kept on the surface so callers
// won't churn when bufio later grows internal allocations.
export fn finish(s: *scanner) void = { };
// readahead — make room and read once from src into the back of the
// pending region. Returns the number of bytes newly buffered (≥0,
// can be 0 if the underlying stream made no progress), or
// io.eof/io.closed propagated from src.
fn readahead(s: *scanner) (i32 | io.eof | io.closed) = {
if (s.start + s.avail == s.cap && s.start > 0) {
// Shift pending region to the front of the buffer.
let i: i32 = 0;
for (i < s.avail) {
s.ptr[i] = s.ptr[s.start + i];
i += 1;
};
s.start = 0;
};
let off: i32 = s.start + s.avail;
let v: []u8;
v.ptr = s.ptr + (off: u64);
v.len = s.cap - off;
let r: (i32 | io.eof | io.closed) = io.read(s.src, v);
match (r) {
case let n: i32 => {
s.avail += n;
return n;
};
case io.eof => { let e: io.eof; return e; };
case io.closed => { let e: io.closed; return e; };
};
};
// scanbyte — pop one byte from the scanner, refilling from src on
// demand. Mirrors Hare's `scan_byte` (ref/hare/bufio/scanner.ha:204).
export fn scanbyte(s: *scanner) (u8 | io.eof | io.closed) = {
for (s.avail == 0) {
let r: (i32 | io.eof | io.closed) = readahead(s);
match (r) {
case let n: i32 => { };
case io.eof => { let e: io.eof; return e; };
case io.closed => { let e: io.closed; return e; };
};
};
let b: u8 = s.ptr[s.start];
s.start += 1;
s.avail -= 1;
return b;
};
// scantok — read up to (and not including) the next byte equal to
// `delim`. The delim byte is consumed from the stream but not
// returned. The returned slice borrows from the scanner's internal
// buffer and is invalidated by the next scan call.
//
// EOF without finding delim discards the trailing fragment and
// returns io.eof (Hare's EOF_DISCARD default). Buffer-full without
// delim returns overflow.
//
// Mirrors Hare's `scan_bytes` (ref/hare/bufio/scanner.ha:220),
// narrowed to a single-byte delimiter.
export fn scantok(s: *scanner, delim: u8) ([]u8 | io.eof | io.closed | overflow) = {
let i: i32 = 0;
for (true) {
for (i < s.avail) {
if (s.ptr[s.start + i] == delim) {
let v: []u8;
v.ptr = s.ptr + (s.start: u64);
v.len = i;
s.start += i + 1;
s.avail -= i + 1;
return v;
};
i += 1;
};
// No delim in pending. If the buffer is full with nowhere
// to shift, the caller's budget is too small — overflow.
if (s.start + s.avail == s.cap && s.start == 0) {
let e: overflow; return e;
};
let r: (i32 | io.eof | io.closed) = readahead(s);
match (r) {
case let n: i32 => {
// readahead may shift start to 0; the searched bytes
// move with it, so `i` (count from start) is still
// accurate. Resume scanning where we left off.
};
case io.eof => { let e: io.eof; return e; };
case io.closed => { let e: io.closed; return e; };
};
};
let e: io.eof; return e;
};
// scanline — read up to (and not including) the next '\n'. The
// newline is consumed; the returned str view borrows from the
// scanner buffer and is invalidated by the next scan call.
// Hare's `scan_line` is `scan_string(s, "\n")`; we route through
// scantok directly since lib/strings doesn't yet ship `toutf8` over
// a multi-byte delim.
export fn scanline(s: *scanner) (str | io.eof | io.closed | overflow) = {
let r: ([]u8 | io.eof | io.closed | overflow) = scantok(s, 10u8);
match (r) {
case let bs: []u8 => {
let v: str;
v.ptr = bs.ptr;
v.len = bs.len;
return v;
};
case io.eof => { let e: io.eof; return e; };
case io.closed => { let e: io.closed; return e; };
case overflow => { let e: overflow; return e; };
};
};
// stream — buffered read+write over an underlying *io.stream.
//
// `vtable` is the first field so a `*stream` is castable to an
// `*io.stream` via address-of-field (`&b.vtable`). Higher layers
// (fmt.fprint, scanner, …) only see the io.stream view; bufio
// recovers the outer stream by casting the dispatch arg back to
// `*stream` inside the bread/bwrite/bclose callbacks.
// stream — heap-free buffered read+write over an underlying io.stream.
// `vt` at offset 0 for the intrusive io.stream→*stream cast. `src` is
// an io.stream (the vtable-native underlying handle). Mirrors
// ref/hare/bufio/stream.ha:18.
//
// rbuf[rstart..rend] is pending read data; wbuf[0..wend] is pending
// write data. rbuf or wbuf may be zero-length: read-empty makes
// bread return io.eof immediately, write-empty makes bwrite a
// pass-through to src (uses src.write directly, no buffering).
// write data. rbuf or wbuf may be zero-length: read-empty makes bread
// return io.eof immediately, write-empty makes bwrite a pass-through to
// src (no buffering).
export type stream = struct {
vtable: io.stream,
src: *io.stream,
vt: io.vtable,
src: io.stream,
rbuf: []u8,
rstart: i32,
rend: i32,
@@ -260,53 +88,57 @@ export type stream = struct {
flush: []u8,
};
// init — wire `b` over `src` with caller-supplied buffers. Both
// rbuf and wbuf may be empty slices; the corresponding direction
// degenerates (see [[stream]]). The flush byte-set defaults to
// "\n" (line-buffered writes); [[setflush]] swaps it.
export fn init(b: *stream, src: *io.stream, rbuf: []u8, wbuf: []u8) void = {
b.vtable.ctx = b: *void;
b.vtable.read = bread;
b.vtable.write = bwrite;
b.vtable.close = bclose;
b.src = src;
b.rbuf = rbuf;
// init — wire a buffered stream over an underlying io.stream with
// caller-supplied read/write buffers. Both rbuf and wbuf may be empty;
// bread / bwrite degenerate (see [[stream]]). The flush byte-set
// defaults to "\n" (line-buffered writes); [[setflush]] swaps it.
//
// Returns the stream BY VALUE; the caller passes `&b.vt` to the io
// dispatchers. Mirrors ref/hare/bufio/stream.ha:69.
export fn init(src: io.stream, rbuf: []u8, wbuf: []u8) stream = {
let r: stream;
r.vt.reader = (&bread): *io.reader;
r.vt.writer = (&bwrite): *io.writer;
r.vt.closer = (&bclose): *io.closer;
r.src = src;
// rstart=rbuf.len, rend=rbuf.len: pre-read unread budget = rbuf.len
// (Hare bufio/stream.ha:101).
b.rstart = rbuf.len;
b.rend = rbuf.len;
b.wbuf = wbuf;
b.wend = 0;
b.flush = flushdefault[0:1];
r.rstart = rbuf.len;
r.rend = rbuf.len;
r.rbuf = rbuf;
r.wbuf = wbuf;
r.wend = 0;
r.flush = flushdefault[0:1];
return r;
};
// setflush — install a new flush byte-set. Any byte from `bs`
// appearing in a write payload triggers an automatic flush after
// the write copies into wbuf. Mirrors Hare's `setflush`
// (ref/hare/bufio/stream.ha:128).
// setflush — install a new flush byte-set. Any byte from `bs` appearing
// in a write payload triggers an automatic flush after the write copies
// into wbuf. Mirrors ref/hare/bufio/stream.ha:128.
export fn setflush(b: *stream, bs: []u8) void = {
b.flush = bs;
};
// flush — drain any pending wbuf data to src. Hare returns
// `(void | io::error)`; ww's io.stream write-side error channel
// is `io.closed` alone, so the return shape narrows.
export fn flush(b: *stream) (void | io.closed) = {
// flush — drain any pending wbuf data to src. Public API AND the
// internal drain called by bwrite / bclose. A 0-byte non-error write
// means the sink made no progress (memio.fixed full) — surfaced as a
// nomem-carried io.error rather than spinning, matching what io.writeall
// would do in Hare. Mirrors ref/hare/bufio/stream.ha.
export fn flush(b: *stream) (void | io.error) = {
if (b.wend == 0) { return; };
let off: i32 = 0;
for (off < b.wend) {
let r: (i32 | io.closed) = io.write(b.src, b.wbuf[off:b.wend]);
let r: (size | io.error) = io.write(b.src, b.wbuf[off:b.wend]);
match (r) {
case let n: i32 => {
if (n == 0) {
// Underlying stream made no progress. Treat as
// closed rather than spin; matches what
// io.writeall would do in Hare.
let e: io.closed; return e;
case let n: size => {
if (n == 0: size) {
let nm: nomem;
let e: io.error = nm;
return e;
};
off += n;
off += n: i32;
};
case io.closed => { let e: io.closed; return e; };
case let e: io.error => return e;
};
};
b.wend = 0;
@@ -314,10 +146,9 @@ export fn flush(b: *stream) (void | io.closed) = {
};
// unread — push `buf` back into the read buffer so the next bread
// returns it first. The bytes must fit in front of the pending
// region (rstart >= buf.len); Hare aborts on overflow, ww does the
// same via rt_abort. Mirrors Hare's `stream_unread`
// (ref/hare/bufio/stream.ha:164).
// returns it first. The bytes must fit in front of the pending region
// (rstart >= buf.len); Hare aborts on overflow, ww does the same via
// rtabort. Mirrors ref/hare/bufio/stream.ha:164.
export fn unread(b: *stream, buf: []u8) void = {
if (b.rstart < buf.len) {
rtabort("bufio.unread: more data than rbuf has room for");
@@ -330,35 +161,43 @@ export fn unread(b: *stream, buf: []u8) void = {
b.rstart -= buf.len;
};
// isbuffered — true when `s` is a [[bufio.stream]]'s embedded
// vtable. Hare's discriminator is callback identity
// (ref/hare/bufio/stream.ha:179); we match the shape directly. The
// read or write callback being bread / bwrite is sufficient (close
// alone isn't unique to bufio).
export fn isbuffered(s: *io.stream) bool = {
if (s.read == bread) { return true; };
if (s.write == bwrite) { return true; };
// isbuffered — true when `s` was returned by [[init]]. Hare's
// discriminator is callback identity (ref/hare/bufio/stream.ha:179).
// Either reader or writer matching is sufficient.
export fn isbuffered(s: io.stream) bool = {
match (s.reader) {
case let r: *io.reader => {
if (r == (&bread): *io.reader) { return true; };
};
case void => { };
};
match (s.writer) {
case let w: *io.writer => {
if (w == (&bwrite): *io.writer) { return true; };
};
case void => { };
};
return false;
};
// ---- vtable callbacks ------------------------------------------------
// ---- buffered-stream vtable callbacks ------------------------------------
fn bread(s: *io.stream, buf: []u8) (i32 | io.eof | io.closed) = {
let b: *stream = s.ctx: *stream;
// No read buffer configured: surface eof immediately rather
// than punch through to src (callers asked for a write-only
// stream, reads against it are a misuse).
if (b.rbuf.len == 0) { let e: io.eof; return e; };
// Empty pending region: refill from src. Reset rstart so
// unread has the full buffer to push back into.
// bread — buffered read. Recover stream via the intrusive cast; refill
// rbuf from src via io.read on empty pending region. Mirrors
// ref/hare/bufio/stream.ha (stream_read).
fn bread(s: io.stream, buf: []u8) (size | io.eof | io.error) = {
let b: *stream = s: *stream;
if (b.rbuf.len == 0) {
let e: io.eof; return e;
};
if (b.rstart >= b.rend) {
b.rstart = 0;
b.rend = 0;
let r: (i32 | io.eof | io.closed) = io.read(b.src, b.rbuf);
let r: (size | io.eof | io.error) = io.read(b.src, b.rbuf);
match (r) {
case let n: i32 => { b.rend = n; };
case let n: size => { b.rend = n: i32; };
case io.eof => { let e: io.eof; return e; };
case io.closed => { let e: io.closed; return e; };
case let e: io.error => return e;
};
};
let avail: i32 = b.rend - b.rstart;
@@ -370,17 +209,18 @@ fn bread(s: *io.stream, buf: []u8) (i32 | io.eof | io.closed) = {
i += 1;
};
b.rstart += n;
return n;
return n: size;
};
fn bwrite(s: *io.stream, buf: []u8) (i32 | io.closed) = {
let b: *stream = s.ctx: *stream;
// No write buffer configured: pass through to src directly.
if (b.wbuf.len == 0) { return io.write(b.src, buf); };
// Scan the write payload for any byte present in b.flush — a
// hit triggers a post-copy flush. Hare uses a labeled break to
// exit both loops on first hit (stream.ha:236-246); ww has no
// labeled break, so we bump both indices past their bounds.
// bwrite — buffered write. wbuf-empty passes through to src; otherwise
// default-flush scan + per-batch copy + post-write conditional flush.
// Labeled-break inlined (ww has no labeled break). Mirrors
// ref/hare/bufio/stream.ha (stream_write).
fn bwrite(s: io.stream, buf: []u8) (size | io.error) = {
let b: *stream = s: *stream;
if (b.wbuf.len == 0) {
return io.write(b.src, buf);
};
let doflush: bool = false;
if (b.flush.len != 0) {
let i: i32 = 0;
@@ -401,10 +241,10 @@ fn bwrite(s: *io.stream, buf: []u8) (i32 | io.closed) = {
for (z < buf.len) {
let avail: i32 = b.wbuf.len - b.wend;
if (avail == 0) {
let r: (void | io.closed) = flush(b);
match (r) {
let fr: (void | io.error) = flush(b);
match (fr) {
case void => { };
case io.closed => { let e: io.closed; return e; };
case let e: io.error => return e;
};
avail = b.wbuf.len;
};
@@ -419,21 +259,162 @@ fn bwrite(s: *io.stream, buf: []u8) (i32 | io.closed) = {
z += n;
};
if (doflush) {
let r: (void | io.closed) = flush(b);
match (r) {
let fr: (void | io.error) = flush(b);
match (fr) {
case void => { };
case io.closed => { let e: io.closed; return e; };
case let e: io.error => return e;
};
};
return buf.len;
return buf.len: size;
};
fn bclose(s: *io.stream) (void | io.closed) = {
let b: *stream = s.ctx: *stream;
let r: (void | io.closed) = flush(b);
match (r) {
// bclose — flush pending wbuf, forward close to src via io.close.
// Caller-owned buffers/src are not freed (drew defer-handle deferral).
fn bclose(s: io.stream) (void | io.error) = {
let b: *stream = s: *stream;
let fr: (void | io.error) = flush(b);
match (fr) {
case void => { };
case io.closed => { let e: io.closed; return e; };
case let e: io.error => return e;
};
let cr: (void | io.error) = io.close(b.src);
match (cr) {
case void => return void;
case let e: io.error => return e;
};
};
// ---- scanner (read-ahead tokenizer over an io.stream src) ----------------
//
// Plain tokenizer — no embedded vtable; driven directly via scanbyte /
// scanbytes / scanline, not through io dispatch. Its `src` is an
// io.stream and refills go through io.read. Value-return constructor
// (Hare newscanner_buf, scanner.ha:92).
//
// ptr/cap kept flat (no `buf: []u8`); the scanner predates struct-held
// slice support.
export type scanner = struct {
src: io.stream,
ptr: *u8,
cap: i32,
start: i32, // index where the pending region starts in ptr
avail: i32, // pending byte count; pending = ptr[start..start+avail]
};
// newscannerbuf — wire a scanner to read through `src` using `buf` as
// the fixed read-ahead window. Returns the scanner BY VALUE. This is
// Hare's newscanner_buf (ref/hare/bufio/scanner.ha:92); the auto-grow
// newscanner is a separate feature (#217).
export fn newscannerbuf(src: io.stream, buf: []u8) scanner = {
let r: scanner;
r.src = src;
r.ptr = buf.ptr;
r.cap = buf.len;
r.start = 0;
r.avail = 0;
return r;
};
// finish — release scanner-owned resources. No-op (buffer is
// caller-owned, src isn't closed); kept on the surface so callers won't
// churn. Mirrors ref/hare/bufio/scanner.ha:110.
export fn finish(s: *scanner) void = { };
// readahead — make room and read once from src into the back of the
// pending region. Returns bytes newly buffered (>=0), or io.eof/io.error
// from src. The size from io.read narrows to i32 (buffer-length type).
fn readahead(s: *scanner) (i32 | io.eof | io.error) = {
if (s.start + s.avail == s.cap && s.start > 0) {
let i: i32 = 0;
for (i < s.avail) {
s.ptr[i] = s.ptr[s.start + i];
i += 1;
};
s.start = 0;
};
let off: i32 = s.start + s.avail;
let v: []u8;
v.ptr = s.ptr + (off: u64);
v.len = s.cap - off;
let r: (size | io.eof | io.error) = io.read(s.src, v);
match (r) {
case let n: size => {
s.avail += n: i32;
return n: i32;
};
case io.eof => { let e: io.eof; return e; };
case let e: io.error => return e;
};
};
// scanbyte — pop one byte, refilling from src on demand. Mirrors
// ref/hare/bufio/scanner.ha:204.
export fn scanbyte(s: *scanner) (u8 | io.eof | io.error) = {
for (s.avail == 0) {
let r: (i32 | io.eof | io.error) = readahead(s);
match (r) {
case let n: i32 => { };
case io.eof => { let e: io.eof; return e; };
case let e: io.error => return e;
};
};
let b: u8 = s.ptr[s.start];
s.start += 1;
s.avail -= 1;
return b;
};
// scanbytes — read up to (and not including) the next byte equal to
// `delim`. The delim is consumed but not returned. The returned slice
// borrows from the scanner buffer and is invalidated by the next scan.
// EOF without delim discards the trailing fragment and returns io.eof
// (Hare EOF_DISCARD default); buffer-full without delim returns
// overflow. Single-byte delim only — Hare's `(u8 | []u8)` multibyte
// form is #217. Mirrors ref/hare/bufio/scanner.ha:220 (narrowed).
export fn scanbytes(s: *scanner, delim: u8) ([]u8 | io.eof | io.error | overflow) = {
let i: i32 = 0;
for (true) {
for (i < s.avail) {
if (s.ptr[s.start + i] == delim) {
let v: []u8;
v.ptr = s.ptr + (s.start: u64);
v.len = i;
s.start += i + 1;
s.avail -= i + 1;
return v;
};
i += 1;
};
if (s.start + s.avail == s.cap && s.start == 0) {
let e: overflow; return e;
};
let r: (i32 | io.eof | io.error) = readahead(s);
match (r) {
case let n: i32 => { };
case io.eof => { let e: io.eof; return e; };
case let e: io.error => return e;
};
};
let e: io.eof; return e;
};
// scanline — read up to (and not including) the next '\n'. The newline
// is consumed; the returned str view borrows from the scanner buffer.
// Single-byte route (Hare's scan_line = scan_string(s, "\n"); the
// arbitrary multibyte-delim scan_string is #217). Mirrors
// ref/hare/bufio/scanner.ha:307.
export fn scanline(s: *scanner) (str | io.eof | io.error | overflow) = {
let r: ([]u8 | io.eof | io.error | overflow) = scanbytes(s, 10u8);
match (r) {
case let bs: []u8 => {
let v: str;
v.ptr = bs.ptr;
v.len = bs.len;
return v;
};
case io.eof => { let e: io.eof; return e; };
case let e: io.error => return e;
case overflow => { let e: overflow; return e; };
};
return io.close(b.src);
};

432
lib/bufio/bufiotest.ww
View File

@@ -1,11 +1,10 @@
// bufiotest — exercises lib/bufio. Run with `out/bin/ww run lib/bufio/bufiotest.ww`.
//
// The scanner @tests enumerate parallel `[N]T` arrays of inputs and
// expectations, then iterate one body across them. Parallel arrays
// (rather than `[N]struct{...}`) match the same cgen workaround
// memiotest leans on. The stream @tests are one scenario per fn
// (the ww-stdlib idiom): the "table" is the fn list in main, not
// a row array.
// expectations, then iterate one body across them. The stream @tests
// are one scenario per fn. #94 fold-eFinal: the unified value-return
// surface — `let st = memio.fixed(buf); &st.vt` into the scanner/stream
// init, io.read/write/close return size/io.error.
package bufio;
@@ -39,38 +38,34 @@ fn putstr(s: str, into: []u8, off: i32) i32 = {
return off + s.len;
};
// ---- A closed-on-read stream for the io.closed surfacing test ---------
// ---- an error-returning stream for the io.error surfacing test --------
//
// Replaces the OLD io.closed source. errvt is module-static; its
// reader/writer return a nomem-widened io.error.
let errvt: io.vtable;
fn closedread(s: *io.stream, buf: []u8) (i32 | io.eof | io.closed) = {
let e: io.closed; return e;
fn errread(s: io.stream, buf: []u8) (size | io.eof | io.error) = {
let nm: nomem; let e: io.error = nm; return e;
};
fn closedwrite(s: *io.stream, buf: []u8) (i32 | io.closed) = {
let e: io.closed; return e;
fn errwrite(s: io.stream, buf: []u8) (size | io.error) = {
let nm: nomem; let e: io.error = nm; return e;
};
fn closedclose(s: *io.stream) (void | io.closed) = { return; };
fn closedstream(s: *io.stream) void = {
s.ctx = nil;
s.read = closedread;
s.write = closedwrite;
s.close = closedclose;
fn errsource() io.stream = {
errvt.reader = (&errread): *io.reader;
errvt.writer = (&errwrite): *io.writer;
return &errvt;
};
// ---- scanbyte: drain four bytes through a 2-byte window ---------------
// Reading buf is smaller than the stream contents (2 vs 4) — the
// scanner has to refill twice. Row 4 is one past EOF, row 5 stays at
// EOF (idempotent).
@test fn scanbytecases() void = {
let raw: [4]u8;
raw[0] = 11u8; raw[1] = 22u8; raw[2] = 33u8; raw[3] = 44u8;
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, raw[0:4]);
let mem: memio.stream = memio.fixed(raw[0:4]);
let m: io.stream = &mem.vt;
let buf: [2]u8;
let sc: bufio.scanner;
bufio.newscanner(&sc, &m, buf[0:2]);
let sc: bufio.scanner = bufio.newscannerbuf(m, buf[0:2]);
// (wantbyte, weof)
let want: [6]u8;
@@ -84,14 +79,14 @@ fn closedstream(s: *io.stream) void = {
let i: i32 = 0;
for (i < 6) {
let r: (u8 | io.eof | io.closed) = bufio.scanbyte(&sc);
let r: (u8 | io.eof | io.error) = bufio.scanbyte(&sc);
match (r) {
case let b: u8 => {
if (weof[i] != 0) { fail(); };
if (b != want[i]) { fail(); };
};
case io.eof => { if (weof[i] == 0) { fail(); }; };
case io.closed => fail();
case let e: io.error => fail();
};
i += 1;
};
@@ -105,17 +100,12 @@ fn closedstream(s: *io.stream) void = {
let src: [32]u8;
let n: i32 = putstr("foo\nbar\nbaz\ntrailing", src[0:32], 0);
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, src[0:n]);
let mem: memio.stream = memio.fixed(src[0:n]);
let m: io.stream = &mem.vt;
let buf: [32]u8;
let sc: bufio.scanner;
bufio.newscanner(&sc, &m, buf[0:32]);
let sc: bufio.scanner = bufio.newscannerbuf(m, buf[0:32]);
// (wantlen, wantfirst, weof, wovr)
// Row 0..2: terminated lines.
// Row 3: tail "trailing" — EOF_DISCARD drops it, returns eof.
// Row 4: subsequent call stays at eof.
let wl: [5]i32;
let wf: [5]u8;
let weof: [5]i32;
@@ -128,7 +118,7 @@ fn closedstream(s: *io.stream) void = {
let i: i32 = 0;
for (i < 5) {
let r: (str | io.eof | io.closed | bufio.overflow) = bufio.scanline(&sc);
let r: (str | io.eof | io.error | bufio.overflow) = bufio.scanline(&sc);
match (r) {
case let v: str => {
if (weof[i] != 0) { fail(); };
@@ -137,7 +127,7 @@ fn closedstream(s: *io.stream) void = {
if (v.len > 0 && v[0] != wf[i]) { fail(); };
};
case io.eof => { if (weof[i] == 0) { fail(); }; };
case io.closed => fail();
case let e: io.error => fail();
case bufio.overflow => { if (wovr[i] == 0) { fail(); }; };
};
i += 1;
@@ -152,47 +142,37 @@ fn closedstream(s: *io.stream) void = {
let src: [32]u8;
let n: i32 = putstr("ABCDEFGHI\n", src[0:32], 0);
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, src[0:n]);
let mem: memio.stream = memio.fixed(src[0:n]);
let m: io.stream = &mem.vt;
// Buffer too small for the 9-byte payload + delim. Caller
// owns the budget; overflow signals "raise it".
// Buffer too small for the 9-byte payload + delim.
let buf: [4]u8;
let sc: bufio.scanner;
bufio.newscanner(&sc, &m, buf[0:4]);
let sc: bufio.scanner = bufio.newscannerbuf(m, buf[0:4]);
let r: (str | io.eof | io.closed | bufio.overflow) = bufio.scanline(&sc);
let r: (str | io.eof | io.error | bufio.overflow) = bufio.scanline(&sc);
match (r) {
case let v: str => fail();
case io.eof => fail();
case io.closed => fail();
case let e: io.error => fail();
case bufio.overflow => { };
};
bufio.finish(&sc);
};
// ---- scantok: multi-delim hit, delim-at-start, EOF before delim ------
// ---- scanbytes: multi-delim hit, delim-at-start, EOF before delim ----
@test fn scantokcases() void = {
@test fn scanbytescases() void = {
let src: [16]u8;
// ",a,,b,c" — leading delim, empty token, multi-hit, trailing fragment.
let n: i32 = putstr(",a,,b,c", src[0:16], 0);
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, src[0:n]);
let mem: memio.stream = memio.fixed(src[0:n]);
let m: io.stream = &mem.vt;
let buf: [8]u8;
let sc: bufio.scanner;
bufio.newscanner(&sc, &m, buf[0:8]);
let sc: bufio.scanner = bufio.newscannerbuf(m, buf[0:8]);
// (wantlen, wantfirst, weof)
// Row 0: leading "," → empty token.
// Row 1: "a" before next ",".
// Row 2: empty token between consecutive ",,".
// Row 3: "b".
// Row 4: "c" — no trailing delim → EOF_DISCARD drops + eof.
let wl: [5]i32;
let wf: [5]u8;
let weof: [5]i32;
@@ -204,7 +184,7 @@ fn closedstream(s: *io.stream) void = {
let i: i32 = 0;
for (i < 5) {
let r: ([]u8 | io.eof | io.closed | bufio.overflow) = bufio.scantok(&sc, 44u8); // ','
let r: ([]u8 | io.eof | io.error | bufio.overflow) = bufio.scanbytes(&sc, 44u8); // ','
match (r) {
case let v: []u8 => {
if (weof[i] != 0) { fail(); };
@@ -212,7 +192,7 @@ fn closedstream(s: *io.stream) void = {
if (v.len > 0 && v[0] != wf[i]) { fail(); };
};
case io.eof => { if (weof[i] == 0) { fail(); }; };
case io.closed => fail();
case let e: io.error => fail();
case bufio.overflow => fail();
};
i += 1;
@@ -225,136 +205,120 @@ fn closedstream(s: *io.stream) void = {
@test fn emptystream() void = {
let raw: [1]u8;
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, raw[0:0]);
let mem: memio.stream = memio.fixed(raw[0:0]);
let m: io.stream = &mem.vt;
let buf: [8]u8;
let sc: bufio.scanner;
bufio.newscanner(&sc, &m, buf[0:8]);
let sc: bufio.scanner = bufio.newscannerbuf(m, buf[0:8]);
let r1: (u8 | io.eof | io.closed) = bufio.scanbyte(&sc);
let r1: (u8 | io.eof | io.error) = bufio.scanbyte(&sc);
match (r1) {
case let b: u8 => fail();
case io.eof => { };
case io.closed => fail();
case let e: io.error => fail();
};
let r2: (str | io.eof | io.closed | bufio.overflow) = bufio.scanline(&sc);
let r2: (str | io.eof | io.error | bufio.overflow) = bufio.scanline(&sc);
match (r2) {
case let v: str => fail();
case io.eof => { };
case io.closed => fail();
case let e: io.error => fail();
case bufio.overflow => fail();
};
let r3: ([]u8 | io.eof | io.closed | bufio.overflow) = bufio.scantok(&sc, 10u8);
let r3: ([]u8 | io.eof | io.error | bufio.overflow) = bufio.scanbytes(&sc, 10u8);
match (r3) {
case let v: []u8 => fail();
case io.eof => { };
case io.closed => fail();
case let e: io.error => fail();
case bufio.overflow => fail();
};
bufio.finish(&sc);
};
// ---- io.closed surfacing on a stream that returns closed --------------
// ---- io.error surfacing on a stream that returns error ----------------
@test fn closedsource() void = {
let m: io.stream;
closedstream(&m);
@test fn errorsource() void = {
let m: io.stream = errsource();
let buf: [8]u8;
let sc: bufio.scanner;
bufio.newscanner(&sc, &m, buf[0:8]);
let sc: bufio.scanner = bufio.newscannerbuf(m, buf[0:8]);
let r1: (u8 | io.eof | io.closed) = bufio.scanbyte(&sc);
let r1: (u8 | io.eof | io.error) = bufio.scanbyte(&sc);
match (r1) {
case let b: u8 => fail();
case io.eof => fail();
case io.closed => { };
case let e: io.error => { };
};
let r2: (str | io.eof | io.closed | bufio.overflow) = bufio.scanline(&sc);
let r2: (str | io.eof | io.error | bufio.overflow) = bufio.scanline(&sc);
match (r2) {
case let v: str => fail();
case io.eof => fail();
case io.closed => { };
case let e: io.error => { };
case bufio.overflow => fail();
};
let r3: ([]u8 | io.eof | io.closed | bufio.overflow) = bufio.scantok(&sc, 32u8);
let r3: ([]u8 | io.eof | io.error | bufio.overflow) = bufio.scanbytes(&sc, 32u8);
match (r3) {
case let v: []u8 => fail();
case io.eof => fail();
case io.closed => { };
case let e: io.error => { };
case bufio.overflow => fail();
};
bufio.finish(&sc);
};
// ---- boundary: empty line + idempotent EOF on scantok ----------------
// ---- boundary: empty line + idempotent EOF on scanbytes --------------
// scanline on a leading '\n' must return an empty (len=0) view, not
// skip the empty line and pretend it's "the first non-empty line".
// scantok after EOF_DISCARD must keep returning io.eof on every
// subsequent call — same idempotency guaranteed by scanbyte rows 4/5
// and scanline rows 3/4, but the scantok path filled at the bottom
// of `for (true)` is its own state machine. Worth pinning.
@test fn boundarycases() void = {
// ---- empty line via scanline -----------------------------------
let s1: [16]u8;
let n1: i32 = putstr("\nfoo\n", s1[0:16], 0);
let m1mem: memio.state;
let m1: io.stream;
memio.fixed(&m1mem, &m1, s1[0:n1]);
let m1mem: memio.stream = memio.fixed(s1[0:n1]);
let m1: io.stream = &m1mem.vt;
let b1: [8]u8;
let sc1: bufio.scanner;
bufio.newscanner(&sc1, &m1, b1[0:8]);
let sc1: bufio.scanner = bufio.newscannerbuf(m1, b1[0:8]);
let r1: (str | io.eof | io.closed | bufio.overflow) = bufio.scanline(&sc1);
let r1: (str | io.eof | io.error | bufio.overflow) = bufio.scanline(&sc1);
match (r1) {
case let v: str => { if (v.len != 0) { fail(); }; };
case io.eof => fail();
case io.closed => fail();
case let e: io.error => fail();
case bufio.overflow => fail();
};
let r2: (str | io.eof | io.closed | bufio.overflow) = bufio.scanline(&sc1);
let r2: (str | io.eof | io.error | bufio.overflow) = bufio.scanline(&sc1);
match (r2) {
case let v: str => {
if (v.len != 3) { fail(); };
if (v[0] != 102u8) { fail(); }; // 'f'
};
case io.eof => fail();
case io.closed => fail();
case let e: io.error => fail();
case bufio.overflow => fail();
};
bufio.finish(&sc1);
// ---- scantok idempotent EOF -----------------------------------
// "ab" with delim ',' — no delim, EOF_DISCARD drops "ab", first
// call returns eof. The next call must too (state stays at eof).
// ---- scanbytes idempotent EOF ---------------------------------
let s2: [8]u8;
let n2: i32 = putstr("ab", s2[0:8], 0);
let m2mem: memio.state;
let m2: io.stream;
memio.fixed(&m2mem, &m2, s2[0:n2]);
let m2mem: memio.stream = memio.fixed(s2[0:n2]);
let m2: io.stream = &m2mem.vt;
let b2: [8]u8;
let sc2: bufio.scanner;
bufio.newscanner(&sc2, &m2, b2[0:8]);
let sc2: bufio.scanner = bufio.newscannerbuf(m2, b2[0:8]);
let t1: ([]u8 | io.eof | io.closed | bufio.overflow) = bufio.scantok(&sc2, 44u8);
let t1: ([]u8 | io.eof | io.error | bufio.overflow) = bufio.scanbytes(&sc2, 44u8);
match (t1) {
case let v: []u8 => fail();
case io.eof => { };
case io.closed => fail();
case let e: io.error => fail();
case bufio.overflow => fail();
};
let t2: ([]u8 | io.eof | io.closed | bufio.overflow) = bufio.scantok(&sc2, 44u8);
let t2: ([]u8 | io.eof | io.error | bufio.overflow) = bufio.scanbytes(&sc2, 44u8);
match (t2) {
case let v: []u8 => fail();
case io.eof => { };
case io.closed => fail();
case let e: io.error => fail();
case bufio.overflow => fail();
};
bufio.finish(&sc2);
@@ -362,20 +326,14 @@ fn closedstream(s: *io.stream) void = {
// ---- multi-fill: window smaller than longest token, shift path --------
// Buffer 4B, line "abcd\nxy\n". After reading "abcd", a delim search
// finds nothing, the buffer is full at start=0 → overflow. So bump
// to 5B (room for "abcd" + '\n'). Reads come in 5-byte chunks; the
// second line "xy" fits trivially and exercises the shift path.
@test fn multifill() void = {
let src: [32]u8;
let n: i32 = putstr("abcd\nxy\n", src[0:32], 0);
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, src[0:n]);
let mem: memio.stream = memio.fixed(src[0:n]);
let m: io.stream = &mem.vt;
let buf: [5]u8;
let sc: bufio.scanner;
bufio.newscanner(&sc, &m, buf[0:5]);
let sc: bufio.scanner = bufio.newscannerbuf(m, buf[0:5]);
// (wantlen, wantfirst, weof)
let wl: [3]i32;
@@ -387,7 +345,7 @@ fn closedstream(s: *io.stream) void = {
let i: i32 = 0;
for (i < 3) {
let r: (str | io.eof | io.closed | bufio.overflow) = bufio.scanline(&sc);
let r: (str | io.eof | io.error | bufio.overflow) = bufio.scanline(&sc);
match (r) {
case let v: str => {
if (weof[i] != 0) { fail(); };
@@ -395,7 +353,7 @@ fn closedstream(s: *io.stream) void = {
if (v.len > 0 && v[0] != wf[i]) { fail(); };
};
case io.eof => { if (weof[i] == 0) { fail(); }; };
case io.closed => fail();
case let e: io.error => fail();
case bufio.overflow => fail();
};
i += 1;
@@ -408,30 +366,28 @@ fn closedstream(s: *io.stream) void = {
@test fn streamsmallwrite() void = {
let raw: [16]u8;
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, raw[0:16]);
let mem: memio.stream = memio.fixed(raw[0:16]);
let m: io.stream = &mem.vt;
let b: bufio.stream;
let rb: [8]u8;
let wb: [8]u8;
bufio.init(&b, &m, rb[0:8], wb[0:8]);
let b: bufio.stream = bufio.init(m, rb[0:8], wb[0:8]);
// Write under the buffer limit — nothing reaches src.
let buf: [5]u8;
let z: i32 = putstr("hello", buf[0:5], 0);
let p: *io.stream = &b.vtable;
let r: (i32 | io.closed) = io.write(p, buf[0:5]);
let p: io.stream = &b.vt;
let r: (size | io.error) = io.write(p, buf[0:5]);
match (r) {
case let n: i32 => { if (n != 5) { fail(); }; };
case io.closed => fail();
case let n: size => { if (n: i32 != 5) { fail(); }; };
case let e: io.error => fail();
};
if (mem.pos != 0) { fail(); };
let fr: (void | io.closed) = bufio.flush(&b);
let fr: (void | io.error) = bufio.flush(&b);
match (fr) {
case void => { };
case io.closed => fail();
case let e: io.error => fail();
};
if (mem.pos != 5) { fail(); };
if (raw[0] != 104u8) { fail(); };
@@ -442,32 +398,29 @@ fn closedstream(s: *io.stream) void = {
@test fn streamautoflush() void = {
let raw: [32]u8;
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, raw[0:32]);
let mem: memio.stream = memio.fixed(raw[0:32]);
let m: io.stream = &mem.vt;
let b: bufio.stream;
let rb: [4]u8;
let wb: [4]u8;
bufio.init(&b, &m, rb[0:4], wb[0:4]);
let b: bufio.stream = bufio.init(m, rb[0:4], wb[0:4]);
// 10B payload through a 4B wbuf: bwrite must flush twice mid-write,
// leaving the final 2B pending until an explicit flush.
// 10B payload through a 4B wbuf: bwrite must flush twice mid-write.
let buf: [10]u8;
let z: i32 = putstr("abcdefghij", buf[0:10], 0);
let p: *io.stream = &b.vtable;
let r: (i32 | io.closed) = io.write(p, buf[0:10]);
let p: io.stream = &b.vt;
let r: (size | io.error) = io.write(p, buf[0:10]);
match (r) {
case let n: i32 => { if (n != 10) { fail(); }; };
case io.closed => fail();
case let n: size => { if (n: i32 != 10) { fail(); }; };
case let e: io.error => fail();
};
// Two full flushes happened (8B); 2B still pending.
if (mem.pos != 8) { fail(); };
let fr: (void | io.closed) = bufio.flush(&b);
let fr: (void | io.error) = bufio.flush(&b);
match (fr) {
case void => { };
case io.closed => fail();
case let e: io.error => fail();
};
if (mem.pos != 10) { fail(); };
if (raw[0] != 97u8) { fail(); }; // 'a'
@@ -476,40 +429,34 @@ fn closedstream(s: *io.stream) void = {
// ---- default flush byte-set: '\n' in payload triggers flush ---------
// init seeds b.flush to ['\n']; bwrite must flush after copying when
// the payload contains any byte from b.flush. No setflush call —
// this exercises the default. Mirrors Hare's flag::NONE default
// (ref/hare/bufio/stream.ha:75 + 100).
@test fn streamlineflush() void = {
let raw: [32]u8;
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, raw[0:32]);
let mem: memio.stream = memio.fixed(raw[0:32]);
let m: io.stream = &mem.vt;
let b: bufio.stream;
let rb: [4]u8;
let wb: [16]u8;
bufio.init(&b, &m, rb[0:4], wb[0:16]);
let b: bufio.stream = bufio.init(m, rb[0:4], wb[0:16]);
let p: *io.stream = &b.vtable;
let p: io.stream = &b.vt;
// First write: no '\n', stays buffered.
let h: [5]u8;
let zh: i32 = putstr("hello", h[0:5], 0);
let r1: (i32 | io.closed) = io.write(p, h[0:5]);
let r1: (size | io.error) = io.write(p, h[0:5]);
match (r1) {
case let n: i32 => { if (n != 5) { fail(); }; };
case io.closed => fail();
case let n: size => { if (n: i32 != 5) { fail(); }; };
case let e: io.error => fail();
};
if (mem.pos != 0) { fail(); };
// Second write: '\n' present, triggers flush of everything (6B).
let nl: [1]u8;
nl[0] = 10u8;
let r2: (i32 | io.closed) = io.write(p, nl[0:1]);
let r2: (size | io.error) = io.write(p, nl[0:1]);
match (r2) {
case let n: i32 => { if (n != 1) { fail(); }; };
case io.closed => fail();
case let n: size => { if (n: i32 != 1) { fail(); }; };
case let e: io.error => fail();
};
if (mem.pos != 6) { fail(); };
if (raw[0] != 104u8) { fail(); };
@@ -518,48 +465,42 @@ fn closedstream(s: *io.stream) void = {
// ---- explicit-flush-per-write: caller drives the drain ---------------
// The previous ww-only FLUSH_ON_WRITE flag is gone; the Hare way
// to drain after every write is a manual flush call. Setflush to
// an empty byte-set first so the default '\n' detector doesn't
// shadow what we're testing.
@test fn streamflushonwrite() void = {
let raw: [32]u8;
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, raw[0:32]);
let mem: memio.stream = memio.fixed(raw[0:32]);
let m: io.stream = &mem.vt;
let b: bufio.stream;
let rb: [4]u8;
let wb: [16]u8;
bufio.init(&b, &m, rb[0:4], wb[0:16]);
let b: bufio.stream = bufio.init(m, rb[0:4], wb[0:16]);
let nilbs: [1]u8;
bufio.setflush(&b, nilbs[0:0]);
let p: *io.stream = &b.vtable;
let p: io.stream = &b.vt;
let h: [2]u8;
h[0] = 65u8; h[1] = 66u8; // "AB"
let r1: (i32 | io.closed) = io.write(p, h[0:2]);
let r1: (size | io.error) = io.write(p, h[0:2]);
match (r1) {
case let n: i32 => { if (n != 2) { fail(); }; };
case io.closed => fail();
case let n: size => { if (n: i32 != 2) { fail(); }; };
case let e: io.error => fail();
};
let f1: (void | io.closed) = bufio.flush(&b);
let f1: (void | io.error) = bufio.flush(&b);
match (f1) {
case void => { };
case io.closed => fail();
case let e: io.error => fail();
};
if (mem.pos != 2) { fail(); };
if (raw[0] != 65u8) { fail(); };
let r2: (i32 | io.closed) = io.write(p, h[0:1]);
let r2: (size | io.error) = io.write(p, h[0:1]);
match (r2) {
case let n: i32 => { if (n != 1) { fail(); }; };
case io.closed => fail();
case let n: size => { if (n: i32 != 1) { fail(); }; };
case let e: io.error => fail();
};
let f2: (void | io.closed) = bufio.flush(&b);
let f2: (void | io.error) = bufio.flush(&b);
match (f2) {
case void => { };
case io.closed => fail();
case let e: io.error => fail();
};
if (mem.pos != 3) { fail(); };
if (raw[2] != 65u8) { fail(); };
@@ -569,17 +510,15 @@ fn closedstream(s: *io.stream) void = {
@test fn streamisbuffered() void = {
let raw: [8]u8;
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, raw[0:8]);
let mem: memio.stream = memio.fixed(raw[0:8]);
let m: io.stream = &mem.vt;
let b: bufio.stream;
let rb: [4]u8;
let wb: [4]u8;
bufio.init(&b, &m, rb[0:4], wb[0:4]);
let b: bufio.stream = bufio.init(m, rb[0:4], wb[0:4]);
if (!bufio.isbuffered(&b.vtable)) { fail(); };
if (bufio.isbuffered(&m)) { fail(); };
if (!bufio.isbuffered(&b.vt)) { fail(); };
if (bufio.isbuffered(m)) { fail(); };
};
// ---- bread + unread: pushed-back bytes come out first ----------------
@@ -588,25 +527,22 @@ fn closedstream(s: *io.stream) void = {
let raw: [8]u8;
let n: i32 = putstr("ABCDEFGH", raw[0:8], 0);
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, raw[0:n]);
let mem: memio.stream = memio.fixed(raw[0:n]);
let m: io.stream = &mem.vt;
let b: bufio.stream;
let rb: [8]u8;
let wb: [4]u8;
bufio.init(&b, &m, rb[0:8], wb[0:4]);
let b: bufio.stream = bufio.init(m, rb[0:8], wb[0:4]);
let p: *io.stream = &b.vtable;
let p: io.stream = &b.vt;
// Pull 3 bytes through bread — fills rbuf from src (8B), serves
// 3 of them; rstart=3 after.
// Pull 3 bytes through bread — fills rbuf from src (8B), serves 3.
let out: [4]u8;
let r1: (i32 | io.eof | io.closed) = io.read(p, out[0:3]);
let r1: (size | io.eof | io.error) = io.read(p, out[0:3]);
match (r1) {
case let z: i32 => { if (z != 3) { fail(); }; };
case let z: size => { if (z: i32 != 3) { fail(); }; };
case io.eof => fail();
case io.closed => fail();
case let e: io.error => fail();
};
if (out[0] != 65u8) { fail(); }; // 'A'
if (out[2] != 67u8) { fail(); }; // 'C'
@@ -616,21 +552,21 @@ fn closedstream(s: *io.stream) void = {
push[0] = 88u8; push[1] = 89u8; // "XY"
bufio.unread(&b, push[0:2]);
let r2: (i32 | io.eof | io.closed) = io.read(p, out[0:2]);
let r2: (size | io.eof | io.error) = io.read(p, out[0:2]);
match (r2) {
case let z: i32 => { if (z != 2) { fail(); }; };
case let z: size => { if (z: i32 != 2) { fail(); }; };
case io.eof => fail();
case io.closed => fail();
case let e: io.error => fail();
};
if (out[0] != 88u8) { fail(); }; // 'X'
if (out[1] != 89u8) { fail(); }; // 'Y'
// Subsequent read returns the original tail.
let r3: (i32 | io.eof | io.closed) = io.read(p, out[0:4]);
let r3: (size | io.eof | io.error) = io.read(p, out[0:4]);
match (r3) {
case let z: i32 => { if (z != 4) { fail(); }; };
case let z: size => { if (z: i32 != 4) { fail(); }; };
case io.eof => fail();
case io.closed => fail();
case let e: io.error => fail();
};
if (out[0] != 68u8) { fail(); }; // 'D'
if (out[3] != 71u8) { fail(); }; // 'G'
@@ -638,34 +574,26 @@ fn closedstream(s: *io.stream) void = {
// ---- scanner over a stream-wrapped src: pre-read unread + scanline --
// Push three bytes into a freshly-init'd stream (rstart=rend=rbuf.len
// post-init, per Hare bufio/stream.ha:101 — the unread budget before
// the first read is the full rbuf), then scan a line through the
// stream. The scanner pulls "XYZ" from rbuf first, refills "hello\n"
// from src, and scanline returns "XYZhello".
@test fn streamscannerunread() void = {
let raw: [16]u8;
let n: i32 = putstr("hello\n", raw[0:16], 0);
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, raw[0:n]);
let mem: memio.stream = memio.fixed(raw[0:n]);
let m: io.stream = &mem.vt;
let b: bufio.stream;
let rb: [16]u8;
let wb: [4]u8;
bufio.init(&b, &m, rb[0:16], wb[0:4]);
let b: bufio.stream = bufio.init(m, rb[0:16], wb[0:4]);
let push: [3]u8;
push[0] = 88u8; push[1] = 89u8; push[2] = 90u8; // "XYZ"
bufio.unread(&b, push[0:3]);
let p: *io.stream = &b.vtable;
let p: io.stream = &b.vt;
let sbuf: [32]u8;
let sc: bufio.scanner;
bufio.newscanner(&sc, p, sbuf[0:32]);
let sc: bufio.scanner = bufio.newscannerbuf(p, sbuf[0:32]);
let lr: (str | io.eof | io.closed | bufio.overflow) = bufio.scanline(&sc);
let lr: (str | io.eof | io.error | bufio.overflow) = bufio.scanline(&sc);
match (lr) {
case let v: str => {
if (v.len != 8) { fail(); }; // "XYZhello"
@@ -673,7 +601,7 @@ fn closedstream(s: *io.stream) void = {
if (v[7] != 111u8) { fail(); }; // 'o'
};
case io.eof => fail();
case io.closed => fail();
case let e: io.error => fail();
case bufio.overflow => fail();
};
bufio.finish(&sc);
@@ -683,55 +611,48 @@ fn closedstream(s: *io.stream) void = {
@test fn streamflushempty() void = {
let raw: [8]u8;
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, raw[0:8]);
let mem: memio.stream = memio.fixed(raw[0:8]);
let m: io.stream = &mem.vt;
let b: bufio.stream;
let rb: [4]u8;
let wb: [4]u8;
bufio.init(&b, &m, rb[0:4], wb[0:4]);
let b: bufio.stream = bufio.init(m, rb[0:4], wb[0:4]);
let r: (void | io.closed) = bufio.flush(&b);
let r: (void | io.error) = bufio.flush(&b);
match (r) {
case void => { };
case io.closed => fail();
case let e: io.error => fail();
};
if (mem.pos != 0) { fail(); };
};
// ---- bclose drains pending wbuf then forwards close to src ----------
// io.close on the stream vtable must run the flush path (Hare
// stream.ha:188-202). Buffer two bytes, route io.close through the
// stream, and confirm both reach src.
@test fn streamcloseflushes() void = {
let raw: [8]u8;
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, raw[0:8]);
let mem: memio.stream = memio.fixed(raw[0:8]);
let m: io.stream = &mem.vt;
let b: bufio.stream;
let rb: [4]u8;
let wb: [4]u8;
bufio.init(&b, &m, rb[0:4], wb[0:4]);
let b: bufio.stream = bufio.init(m, rb[0:4], wb[0:4]);
let nilbs: [1]u8;
bufio.setflush(&b, nilbs[0:0]); // suppress the default '\n' detector
let p: *io.stream = &b.vtable;
let p: io.stream = &b.vt;
let h: [2]u8;
h[0] = 80u8; h[1] = 81u8; // "PQ"
let r: (i32 | io.closed) = io.write(p, h[0:2]);
let r: (size | io.error) = io.write(p, h[0:2]);
match (r) {
case let n: i32 => { if (n != 2) { fail(); }; };
case io.closed => fail();
case let n: size => { if (n: i32 != 2) { fail(); }; };
case let e: io.error => fail();
};
if (mem.pos != 0) { fail(); }; // still buffered
let cr: (void | io.closed) = io.close(p);
let cr: (void | io.error) = io.close(p);
match (cr) {
case void => { };
case io.closed => fail();
case let e: io.error => fail();
};
if (mem.pos != 2) { fail(); };
if (raw[0] != 80u8) { fail(); };
@@ -740,34 +661,27 @@ fn closedstream(s: *io.stream) void = {
// ---- setflush with a custom non-empty byte-set -----------------------
// setflush installs a fresh byte-set (not just clears it). Use ' '
// (space) as the trigger; "ab cd" must flush at the space, leaving
// "cd" pending.
@test fn streamsetflushcustom() void = {
let raw: [16]u8;
let mem: memio.state;
let m: io.stream;
memio.fixed(&mem, &m, raw[0:16]);
let mem: memio.stream = memio.fixed(raw[0:16]);
let m: io.stream = &mem.vt;
let b: bufio.stream;
let rb: [4]u8;
let wb: [16]u8;
bufio.init(&b, &m, rb[0:4], wb[0:16]);
let b: bufio.stream = bufio.init(m, rb[0:4], wb[0:16]);
let bs: [1]u8;
bs[0] = 32u8; // ' '
bufio.setflush(&b, bs[0:1]);
let p: *io.stream = &b.vtable;
let p: io.stream = &b.vt;
let buf: [5]u8;
let z: i32 = putstr("ab cd", buf[0:5], 0);
let r: (i32 | io.closed) = io.write(p, buf[0:5]);
let r: (size | io.error) = io.write(p, buf[0:5]);
match (r) {
case let n: i32 => { if (n != 5) { fail(); }; };
case io.closed => fail();
case let n: size => { if (n: i32 != 5) { fail(); }; };
case let e: io.error => fail();
};
// Space at index 2 triggers post-copy flush of all 5 bytes; the
// scan-then-copy-then-flush ordering writes everything, so mem.pos
// hits 5 in one shot.
// Space at index 2 triggers post-copy flush of all 5 bytes.
if (mem.pos != 5) { fail(); };
if (raw[2] != 32u8) { fail(); };
if (raw[4] != 100u8) { fail(); }; // 'd'
@@ -777,9 +691,9 @@ export fn main() i32 = {
signalled = 1; scanbytecases();
signalled = 2; scanlinecases();
signalled = 3; scanlineoverflow();
signalled = 4; scantokcases();
signalled = 4; scanbytescases();
signalled = 5; emptystream();
signalled = 6; closedsource();
signalled = 6; errorsource();
signalled = 7; boundarycases();
signalled = 8; multifill();
signalled = 9; streamsmallwrite();

429
lib/bufio/vstream.ww
View File

@@ -1,429 +0,0 @@
// vstream — Hare-shaped vtable port of lib/bufio. Project #94
// fold-eFinal(PREP).
//
// Collapses the OLD bufio surface (bufio.ww's pre-vtable `stream` +
// `scanner` over a legacy `*io.stream` src) onto the io.vtable surface,
// as parallel `_v`-suffixed additions. The OLD surface stays untouched
// here — fold-eFinal (FLIP) deletes it, renames the `_v` suffix off
// (bufio_ctx→stream, bufio_vstream→init, scanner_v→scanner, scanbytes_v
// →scanbytes, isbuffered_v→isbuffered, …) per Hare's
// ref/hare/bufio/{stream,scanner}.ha, and migrates the few callers.
//
// SCOPE (#94 fold-eFinal): this is a COLLAPSE of the surface ww ALREADY
// ships, NOT a port of Hare's full scanner. The features OLD bufio
// never implemented — auto-grow newscanner, multibyte-delim scanbytes /
// scanstring, scanrune / readbyte / readtok / readline / readrune /
// unreadrune — are a separate future feature fold (#217) and are OUT of
// eFinal. The ported set mirrors OLD exactly: the fixed-buffer
// newscanner_buf, single-byte scanbytes (OLD scantok), single-byte
// scanline, scanbyte, finish; and the buffered-stream init / setflush /
// flush / unread / isbuffered.
//
// VALUE-RETURN (drew, gating): Hare's bufio constructors return BY
// VALUE (ref/hare/bufio/stream.ha:69 init, scanner.ha:92
// newscanner_buf). The OLD out-param shape existed only because
// wide-struct sret was unimplemented — it now round-trips
// (test/wcc/925 + 776/778 byte-id), so each constructor builds in a
// local `let r: T;`, field-assigns every slot, and `return r;`. The
// caller owns the returned struct (stack ownership, no-GC). For the
// buffered stream the caller passes `&b.vt` to the io.st_* dispatchers;
// the scanner is a plain read-ahead tokenizer (no embedded vtable —
// matching OLD), driven directly via scanbyte_v / scanbytes_v / …
//
// VTABLE-NATIVE SRC (the eFinal point): the buffered stream's `src` and
// the scanner's `src` are now `io.vstream`, and every underlying read /
// write / close goes through io.st_read / io.st_write / io.st_close
// (the vtable dispatchers) — NOT the legacy io.read / io.write /
// io.close over `*io.stream`. The io.st_* surface returns io.error
// directly, so the OLD `io.closed → nomem-widen → io.error` dance
// (per the #173 deferral) is GONE: the error simply forwards.
//
// Cast workaround per #206-payoff (ken: KEEP the explicit casts; they
// are cgen-neutral and sidestep the #214 over-acceptance surface).
// `(&fn_name): *io.<role>` at each vtable store + the isbuffered_v
// fn-ptr-equality comparand — same workaround memio/vstream.ww +
// test/wcc/775 use.
//
// #207 (struct-lit multi-tagged-field copy drops past first) and #210
// (slice-field struct-lit drop under alloc): not reachable from the
// value-return form — every struct is built field-by-field in a local
// then sret-returned, no `alloc(T{...})` struct-lit. Both filings stay
// open for the struct-lit path; this fold doesn't take it.
//
// #209 (wwstage formattable match-arm bail when fmt imported): bufio
// does NOT import fmt (only io), so test/wcc/778 runs both stages +
// cs.s == ww.s byte-id on every row — bufio's only byte-id coverage
// (it is not compiler-embedded).
//
// Mode discrimination (drew-deferred): Hare uses three vtable
// singletons (vtable_r / vtable_w / vtable_rw); bufio_vstream installs
// all three callbacks per bufio.ww:32-38's divergence (zero-length
// rbuf/wbuf degenerates the matching callback in-cb). Defer-handle
// (MANAGED_* ownership bits) also deferred — caller owns rbuf/wbuf/src;
// bclose_v flushes + forwards close but frees nothing. Both graduate
// with io fold-2 (#5).
package bufio;
import io;
// bufio_ctx — heap-free state for bufio_vstream. `vt` at offset 0 for
// the intrusive vstream→*bufio_ctx cast. `src` is an io.vstream (the
// vtable-native underlying handle), not the OLD `*io.stream`. Mirrors
// lib/bufio.stream (bufio.ww:252) modulo io.vtable + io.vstream src.
export type bufio_ctx = struct {
vt: io.vtable,
src: io.vstream,
rbuf: []u8,
rstart: i32,
rend: i32,
wbuf: []u8,
wend: i32,
flush: []u8,
};
// bufio_vstream — wire a buffered stream over an underlying io.vstream
// with caller-supplied read/write buffers. Both rbuf and wbuf may be
// empty; bread_v / bwrite_v degenerate per bufio.ww:34-38. The flush
// byte-set defaults to "\n" (line-buffered writes); setflush_v swaps it.
//
// Returns the ctx BY VALUE (sret — field-by-field build then
// `return r;`); the caller passes `&b.vt` to the io.st_* dispatchers.
//
// Mirrors ref/hare/bufio/stream.ha:69 (init).
export fn bufio_vstream(src: io.vstream, rbuf: []u8, wbuf: []u8) bufio_ctx = {
let r: bufio_ctx;
r.vt.reader = (&bread_v): *io.reader;
r.vt.writer = (&bwrite_v): *io.writer;
r.vt.closer = (&bclose_v): *io.closer;
r.src = src;
r.rbuf = rbuf;
r.rstart = rbuf.len;
r.rend = rbuf.len;
r.wbuf = wbuf;
r.wend = 0;
r.flush = flushdefault[0:1];
return r;
};
// setflush_v — install a new flush byte-set. Any byte from `bs`
// appearing in a write payload triggers an automatic flush after the
// write copies into wbuf. Mirrors OLD setflush (bufio.ww:287) +
// ref/hare/bufio/stream.ha:128.
export fn setflush_v(b: *bufio_ctx, bs: []u8) void = {
b.flush = bs;
};
// flush_v — drain any pending wbuf data to src. Public API AND the
// internal drain called by bwrite_v / bclose_v — OLD bufio.flush is
// itself dual-purpose (bufio.ww:294, called from bwrite/bclose), so a
// single fn serves both; no separate private callback to rename. The
// io.closed → nomem-widen of the OLD V-side is gone: io.st_write
// returns io.error directly. A 0-byte non-error write means the sink
// made no progress (memio.fixed full) — surfaced as a nomem-carried
// io.error rather than spinning, matching what io.writeall would do in
// Hare. Mirrors OLD flush (bufio.ww:294) + ref/hare/bufio/stream.ha.
export fn flush_v(b: *bufio_ctx) (void | io.error) = {
if (b.wend == 0) { return; };
let off: i32 = 0;
for (off < b.wend) {
let r: (size | io.error) = io.st_write(b.src, b.wbuf[off:b.wend]);
match (r) {
case let n: size => {
if (n == 0: size) {
let nm: nomem;
let e: io.error = nm;
return e;
};
off += n: i32;
};
case let e: io.error => return e;
};
};
b.wend = 0;
return;
};
// unread_v — push `buf` back into the read buffer so the next bread_v
// returns it first. The bytes must fit in front of the pending region
// (rstart >= buf.len); Hare aborts on overflow, ww does the same via
// rtabort (declared in bufio.ww, same package). Mirrors OLD unread
// (bufio.ww:321) + ref/hare/bufio/stream.ha:164.
export fn unread_v(b: *bufio_ctx, buf: []u8) void = {
if (b.rstart < buf.len) {
rtabort("bufio.unread_v: more data than rbuf has room for");
};
let i: i32 = 0;
for (i < buf.len) {
b.rbuf[b.rstart - buf.len + i] = buf[i];
i += 1;
};
b.rstart -= buf.len;
};
// isbuffered_v — true when `s` was returned by [[bufio_vstream]].
// Hare uses callback-identity (ref/hare/bufio/stream.ha:179); matches
// the OLD isbuffered shape (bufio.ww:338) over the new `_v` symbols.
// Either reader or writer matching is sufficient.
export fn isbuffered_v(s: io.vstream) bool = {
match (s.reader) {
case let r: *io.reader => {
if (r == (&bread_v): *io.reader) { return true; };
};
case void => { };
};
match (s.writer) {
case let w: *io.writer => {
if (w == (&bwrite_v): *io.writer) { return true; };
};
case void => { };
};
return false;
};
// ---- buffered-stream vtable callbacks ------------------------------------
// bread_v — vstream-side buffered read. Recover ctx via the intrusive
// cast; refill rbuf from src via io.st_read on empty pending region.
// Mirrors bufio.ww:346 (OLD bread) modulo the io.vstream src +
// io.st_read dispatch (io.error forwards directly — no nomem-widen).
fn bread_v(s: io.vstream, buf: []u8) (size | io.eof | io.error) = {
let b: *bufio_ctx = s: *bufio_ctx;
if (b.rbuf.len == 0) {
let e: io.eof; return e;
};
if (b.rstart >= b.rend) {
b.rstart = 0;
b.rend = 0;
let r: (size | io.eof | io.error) = io.st_read(b.src, b.rbuf);
match (r) {
case let n: size => { b.rend = n: i32; };
case io.eof => { let e: io.eof; return e; };
case let e: io.error => return e;
};
};
let avail: i32 = b.rend - b.rstart;
let n: i32 = buf.len;
if (avail < n) { n = avail; };
let i: i32 = 0;
for (i < n) {
buf[i] = b.rbuf[b.rstart + i];
i += 1;
};
b.rstart += n;
return n: size;
};
// bwrite_v — vstream-side buffered write. wbuf-empty passes through to
// src; otherwise default-flush scan + per-batch copy + post-write
// conditional flush. Mirrors bufio.ww:376 (OLD bwrite) modulo the
// io.vstream src + io.st_write dispatch. Labeled-break inlined per
// bufio.ww:382 (ww has no labeled break).
fn bwrite_v(s: io.vstream, buf: []u8) (size | io.error) = {
let b: *bufio_ctx = s: *bufio_ctx;
if (b.wbuf.len == 0) {
return io.st_write(b.src, buf);
};
let doflush: bool = false;
if (b.flush.len != 0) {
let i: i32 = 0;
for (i < buf.len) {
let j: i32 = 0;
for (j < b.flush.len) {
if (buf[i] == b.flush[j]) {
doflush = true;
i = buf.len;
j = b.flush.len;
};
j += 1;
};
i += 1;
};
};
let z: i32 = 0;
for (z < buf.len) {
let avail: i32 = b.wbuf.len - b.wend;
if (avail == 0) {
let fr: (void | io.error) = flush_v(b);
match (fr) {
case void => { };
case let e: io.error => return e;
};
avail = b.wbuf.len;
};
let n: i32 = buf.len - z;
if (avail < n) { n = avail; };
let i: i32 = 0;
for (i < n) {
b.wbuf[b.wend + i] = buf[z + i];
i += 1;
};
b.wend += n;
z += n;
};
if (doflush) {
let fr: (void | io.error) = flush_v(b);
match (fr) {
case void => { };
case let e: io.error => return e;
};
};
return buf.len: size;
};
// bclose_v — flush pending wbuf, forward close to src via io.st_close.
// Mirrors bufio.ww:431. Caller-owned buffers/src are not freed (drew
// defer-handle deferral, bufio.ww:33-47).
fn bclose_v(s: io.vstream) (void | io.error) = {
let b: *bufio_ctx = s: *bufio_ctx;
let fr: (void | io.error) = flush_v(b);
match (fr) {
case void => { };
case let e: io.error => return e;
};
let cr: (void | io.error) = io.st_close(b.src);
match (cr) {
case void => return void;
case let e: io.error => return e;
};
};
// ---- scanner (read-ahead tokenizer over an io.vstream src) ---------------
//
// Plain tokenizer — no embedded vtable (matching OLD bufio.scanner,
// bufio.ww:104); the scanner is driven directly via scanbyte_v /
// scanbytes_v / scanline_v, not through io.st_*. Its `src` is an
// io.vstream and refills go through io.st_read. Value-return
// constructor (Hare newscanner_buf, scanner.ha:92).
//
// ptr/cap kept flat (no `buf: []u8`) per the OLD scanner shape
// (bufio.ww:104-110); the scanner predates struct-held slice support.
export type scanner_v = struct {
src: io.vstream,
ptr: *u8,
cap: i32,
start: i32, // index where the pending region starts in ptr
avail: i32, // pending byte count; pending = ptr[start..start+avail]
};
// newscannerbuf_v — wire a scanner to read through `src` using `buf` as
// the fixed read-ahead window. Returns the scanner BY VALUE. This is
// Hare's newscanner_buf (ref/hare/bufio/scanner.ha:92); the auto-grow
// newscanner is a separate feature (#217, OUT of eFinal). Mirrors OLD
// newscanner (bufio.ww:116), which is itself the fixed-buffer form.
export fn newscannerbuf_v(src: io.vstream, buf: []u8) scanner_v = {
let r: scanner_v;
r.src = src;
r.ptr = buf.ptr;
r.cap = buf.len;
r.start = 0;
r.avail = 0;
return r;
};
// finish_v — release scanner-owned resources. No-op (buffer is
// caller-owned, src isn't closed); kept on the surface so callers won't
// churn. Mirrors OLD finish (bufio.ww:127) + scanner.ha:110.
export fn finish_v(s: *scanner_v) void = { };
// readahead_v — make room and read once from src into the back of the
// pending region. Returns bytes newly buffered (>=0), or io.eof/io.error
// from src. Mirrors OLD readahead (bufio.ww:133) modulo io.st_read +
// io.error (no nomem-widen). Returns i32 (OLD shape); the size from
// io.st_read narrows to i32 (buffer-length type).
fn readahead_v(s: *scanner_v) (i32 | io.eof | io.error) = {
if (s.start + s.avail == s.cap && s.start > 0) {
let i: i32 = 0;
for (i < s.avail) {
s.ptr[i] = s.ptr[s.start + i];
i += 1;
};
s.start = 0;
};
let off: i32 = s.start + s.avail;
let v: []u8;
v.ptr = s.ptr + (off: u64);
v.len = s.cap - off;
let r: (size | io.eof | io.error) = io.st_read(s.src, v);
match (r) {
case let n: size => {
s.avail += n: i32;
return n: i32;
};
case io.eof => { let e: io.eof; return e; };
case let e: io.error => return e;
};
};
// scanbyte_v — pop one byte, refilling from src on demand. Mirrors OLD
// scanbyte (bufio.ww:160) + ref/hare/bufio/scanner.ha:204.
export fn scanbyte_v(s: *scanner_v) (u8 | io.eof | io.error) = {
for (s.avail == 0) {
let r: (i32 | io.eof | io.error) = readahead_v(s);
match (r) {
case let n: i32 => { };
case io.eof => { let e: io.eof; return e; };
case let e: io.error => return e;
};
};
let b: u8 = s.ptr[s.start];
s.start += 1;
s.avail -= 1;
return b;
};
// scanbytes_v — read up to (and not including) the next byte equal to
// `delim`. The delim is consumed but not returned. The returned slice
// borrows from the scanner buffer and is invalidated by the next scan.
// EOF without delim discards the trailing fragment and returns io.eof
// (Hare EOF_DISCARD default); buffer-full without delim returns
// overflow. Single-byte delim only — Hare's `(u8 | []u8)` multibyte
// form is #217 (OUT). drew renames OLD scantok → scanbytes. Mirrors OLD
// scantok (bufio.ww:186) + ref/hare/bufio/scanner.ha:220 (narrowed).
export fn scanbytes_v(s: *scanner_v, delim: u8) ([]u8 | io.eof | io.error | overflow) = {
let i: i32 = 0;
for (true) {
for (i < s.avail) {
if (s.ptr[s.start + i] == delim) {
let v: []u8;
v.ptr = s.ptr + (s.start: u64);
v.len = i;
s.start += i + 1;
s.avail -= i + 1;
return v;
};
i += 1;
};
if (s.start + s.avail == s.cap && s.start == 0) {
let e: overflow; return e;
};
let r: (i32 | io.eof | io.error) = readahead_v(s);
match (r) {
case let n: i32 => { };
case io.eof => { let e: io.eof; return e; };
case let e: io.error => return e;
};
};
let e: io.eof; return e;
};
// scanline_v — read up to (and not including) the next '\n'. The newline
// is consumed; the returned str view borrows from the scanner buffer.
// Single-byte route (Hare's scan_line = scan_string(s, "\n"); the
// arbitrary multibyte-delim scan_string is #217, OUT). Mirrors OLD
// scanline (bufio.ww:225) + ref/hare/bufio/scanner.ha:307.
export fn scanline_v(s: *scanner_v) (str | io.eof | io.error | overflow) = {
let r: ([]u8 | io.eof | io.error | overflow) = scanbytes_v(s, 10u8);
match (r) {
case let bs: []u8 => {
let v: str;
v.ptr = bs.ptr;
v.len = bs.len;
return v;
};
case io.eof => { let e: io.eof; return e; };
case let e: io.error => return e;
case overflow => { let e: overflow; return e; };
};
};