Hojun-Cho/ww
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

lib/regex: fold-3 — anchors, escape, postfix ?/*/+, alternation

Browse Source 
Ports compile()'s fold-3 arms (ref/hare/regex/regex.ha): \ escape
(286-293), ^ (294-300), $ (301-312, goes live with run_thread's
anchored ha:621-624 pin), | (335-367) over the restored jump_idxs
prologue (241-256 subset: jump_idxs + was_prev_rune_pipe +
group_level verbatim-but-0) + the whole-expression fixup (470-473,
the SIZE_MAX-sentinel overwrite 2a dropped), ? (403-420), * (421-443),
+ (444-459); find_last_groupstart (104-119, whole — error arm is the
live one until the group fold) and shift (123-133). run_thread's
inst_split/inst_jump aborts FLIP live (606-611, all 2b-proven
shapes). Still loud: [ ( ) { in compile; groupstart/groupend/repeat/
charset arms; add_thread's capture-dup bound.

Spelling divergences, each cited at site:
- multi-type case (inst_lit | inst_charset | inst_any) (ha:407) is
  loud-rejected both stages (PE5; Hare-parity #13) -> three void arms.
- find_last_groupstart(...)? in the dead groupend arms hits the #38b
  >32B-tagged-return propagate loud-stop -> explicit D13 match
  (harec's own ? desugaring).
- by-value range over an indexed element (for (let x .. jump_idxs[g]))
  SEGFAULTS both stages (NEW, filed ww-core #57) -> D9 index loops at
  ha:351-356 + 470-473.
- assert(cond, msg) is wwstage-broken (CALL assert(SB), undefined at
  link; NEW, filed ww-core #58) -> if+abort at the ha:355 site.
- Hare's match/if EXPRESSIONS (ha:337-346) -> statement spellings
  (the #51 search/scanrune precedent).

Tests: the 11-pattern loud row shrinks to the 4 remaining metachars;
strerror reroutes through a[. New: fold3_programs (exact inst
sequences incl. the a|b sentinel fixup), fold3_compile_errors (12
exact texts incl. Hare's own ab\|^cd ERROR fixture), direct
find_last_groupstart/shift rows, fold3_find_cases (42 rows from
Hare's +test.ha anchors/postfix/alternation blocks, group-free
subset, end=-1 resolved to rune-length; rob's dedup/leftmost/longest
riders now observable: a* over aaaa -> single (0,4), b+ over abab ->
(1,2), b* / ^b* over aaaabbbb -> (4,8)/(0,0); multibyte b+ row keeps
idx != bytesize) + the a*-over-baa findall rider (longest-pick beats
the zero-length candidate; trailing zero-length match takes the
ha:942-945 break). Both drivers green; regex_test.combined.ww cs==ww
byte-identical.
This commit is contained in:
Hojun-Cho
2026-06-04 17:06:31 +09:00
parent 9861f73bbb
commit da48b29a7c
2 changed files with 589 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

301
lib/regex/regex.ww
View File

@@ -7,10 +7,13 @@
// tranche C = search, the first end-to-end match; fold 2b tranche D
// = the exec surface (test/find — the D13 explicit-match spelling of
// Hare's multi-success `?`, ww-core #14); fold 2c =
// findall/result_freeall over the memio seeker. Every other
// metacharacter arm (and replace) is DEFERRED behind compiler fixes —
// probed pre-port, pA*/pB*/PB*/PC*/PD* probes. They land with those
// fixes.
// findall/result_freeall over the memio seeker; fold 3 = anchors
// `^`/`$`, the `\` escape, postfix `?`/`*`/`+`, alternation `|`
// (jump_idxs state + find_last_groupstart/shift + the insert()
// builtin) and the run_thread split/jump arms. Remaining
// metacharacter arms (`[` bracket, `(`/`)` groups, `{` repetition —
// and replace) are DEFERRED — probed pre-port, pA*/pB*/PB*/PC*/PD*/
// PE* probes. They land with their folds.
//
// One fold-1 construct is held back behind a filed compiler/fidelity
// gap (see the charclass_map site below):
@@ -132,16 +135,65 @@ export fn finish(re: *regex) void = {
free(re.charsets);
};
// ref/hare/regex/regex.ha:104-119, verbatim. The error arm is ALWAYS
// taken until the group fold ('(' is loud, no inst_groupstart can
// exist); the '|' arm consumes it as origin = 0 (whole-expression
// alternation). Match-on-indexed scrutinee binds through a typed let
// (the run_thread spelling).
fn find_last_groupstart(insts: []inst) (size | error) = {
let nested: uint = 0;
for (let i: size = (len(insts): size); i > 0; i -= 1) {
let cur: inst = insts[i - 1];
match (cur) {
case inst_groupstart => {
if (nested == 0) {
return i - 1;
};
nested -= 1;
};
case inst_groupend => { nested += 1; };
case => void;
};
};
return "Unmatched ')'": error;
};
// Increments all inst_jump and inst_split instructions to account
// for a newly inserted instruction before the given slice.
//
// ref/hare/regex/regex.ha:123-133. Hare's by-ref `&..` range has no
// ww spelling (D9, the add_thread/search precedent) — index loop;
// the write-through is a tagged element store via index (PE3), and
// the sub-slice arg at the call sites (`shift(insts[k:])`) is a
// borrowed VIEW, so the stores land in the caller's backing (PE4).
fn shift(sl: []inst) void = {
for (let i: size = 0; i < (len(sl): size); i += 1) {
let cur: inst = sl[i];
match (cur) {
case let z: inst_jump =>
sl[i] = (((z: size) + 1): inst_jump);
case let z: inst_split =>
sl[i] = (((z: size) + 1): inst_split);
case => void;
};
};
};
// Compiles a regular expression string into a [[regex]].
//
// ref/hare/regex/regex.ha:227-263. Fold 2a ports the literal core:
// inst_lit / inst_any / inst_match plus the leading inst_skip
// (regex.ha:261-263 — unanchored exec depends on it). Every other
// metacharacter arm is one loud not-yet-ported error — the explicit
// fold boundary; falling to literal would be a silent semantic lie.
// State serving only the deferred arms is dropped with them:
// jump_idxs (ha:241-248), the bracket quad (ha:249-252),
// was_prev_rune_pipe / group_level / capture_idx (ha:253-256).
// ref/hare/regex/regex.ha:227-263 + the fold-3 arms: `\` escape
// (ha:286-293), anchors `^` (294-300) / `$` (301-312), alternation
// `|` (335-367) over the jump_idxs state (241-248 subset) +
// whole-expression fixup (470-473), postfix `?` (403-420) / `*`
// (421-443) / `+` (444-459). The remaining metacharacter arms (`[`
// bracket, `(`/`)` groups, `{` repetition) are one loud
// not-yet-ported error — the explicit fold boundary; falling to
// literal would be a silent semantic lie. group_level (ha:255) is
// kept verbatim but only ever 0 until the group fold — the anchors'
// group_level arms and the postfix inst_groupend/groupstart arms are
// dead-but-verbatim. capture_idx and the bracket quad stay dropped
// with their arms; the loop-head `done` Unmatched-'(' check
// (ha:278-280) rides the '(' arm.
//
// Hare's `defer if (!ok) free(...)` cleanup (ha:231-237) is omitted:
// ww has no `defer if` (cf lib/strings/strings.ww:85), and the free()
@@ -155,7 +207,16 @@ export fn compile(expr: str) (regex | error | nomem) = {
let charsets: []charset; // stays empty until the '[' fold
let iter: strings.iterator = strings.iter(expr);
let r_idx: size = 0;
// jump_idxs tracks the pending alternation jumps per group level;
// only level 0 can populate until the group fold. Hare's
// `append(jump_idxs, [])` (ha:242) spells through a typed empty
// let (the #25/#31 ruling; cf the inst_skip let below).
let jump_idxs: [][]size;
let lvl0: []size;
append(jump_idxs, lvl0);
let was_prev_rune_pipe: bool = false;
let n_reps: size = 0;
let group_level: size = 0;
for (true) {
let next: (rune | utf8.done) = strings.next(&iter);
@@ -172,14 +233,184 @@ export fn compile(expr: str) (regex | error | nomem) = {
append(insts, v);
};
// regex.ha:277-284 minus the group_level check (it rides
// the deferred '(' arm's state).
let r: rune = match (next) {
case utf8.done => break;
case let x: rune => yield x;
};
switch (r) {
case '\\': { // regex.ha:286-293
let peek1: (rune | utf8.done) = strings.next(&iter);
if (peek1 is utf8.done) {
return "Trailing backslash '\\'": error;
};
append(insts, ((peek1 as rune): inst_lit));
r_idx += 1;
};
case '^': { // regex.ha:294-300
if (group_level > 0) {
return "Anchor '^' in capture groups is unsupported": error;
};
if (!(r_idx == 0 || was_prev_rune_pipe)) {
return "Anchor '^' not at start of whole pattern or alternation": error;
};
};
case '$': { // regex.ha:301-312
if (group_level > 0) {
return "Anchor '$' in capture groups is unsupported": error;
};
let peek1: (rune | utf8.done) = strings.next(&iter);
if (peek1 is rune) {
if ((peek1 as rune) != '|') {
return "Anchor '$' not at end of whole pattern or alternation": error;
};
strings.prev(&iter);
};
append(insts, (true: inst_match));
};
case '|': { // regex.ha:335-367
append(insts, (types.SIZE_MAX: inst_jump));
// origin = the instruction after the innermost
// groupstart, or 0 for whole-expression alternation
// (the error arm — always taken until the group
// fold). Hare binds via a match EXPRESSION
// (ha:337-342); statement-match into a declared
// local, the search/scanrune precedent (#51).
let origin: size = 0;
match (find_last_groupstart(insts)) {
case let e: error => { origin = 0; };
case let sz: size => { origin = sz + 1; };
};
let newinst: inst = (((insts.len: size) + 1): inst_split);
// add split after last jump (if any) or at origin.
// Hare's if-EXPRESSION (ha:345-346) spells as a
// statement; the tail read is the PE2 double-index.
let split_idx: size = origin;
if (jump_idxs[group_level].len > 0) {
split_idx = jump_idxs[group_level][jump_idxs[group_level].len - 1] + 1;
};
insert(insts[split_idx], newinst);
shift(insts[split_idx + 1:]);
// our insertion of our split_idx should never
// interfere with an existing jump_idx; if this check
// ends up being hit in the future, it is a sign that
// jump_idx should be incremented. Hare asserts
// (ha:351-356); the assert builtin is wwstage-broken
// (ww-core #58) and the by-value range over an
// indexed element segfaults (ww-core #57) — if+abort
// over a D9-class index loop.
for (let k: size = 0; k < (jump_idxs[group_level].len: size); k += 1) {
if (!(jump_idxs[group_level][k] < split_idx)) {
abort("Found jump_idx interference. Please report this as a bug");
};
};
append(jump_idxs[group_level], (insts.len: size) - 1);
// add skip if it's a whole-expression alternation
if (origin == 0) {
let peek1: (rune | utf8.done) = strings.next(&iter);
if (peek1 is rune) {
if ((peek1 as rune) != '^') {
let sk: inst_skip;
let v: inst = sk;
append(insts, v);
};
strings.prev(&iter);
};
};
};
case '?': { // regex.ha:403-420
if (r_idx == 0 || insts.len == 0) {
return "Unused '?'": error;
};
let term_start_idx: size = (insts.len: size) - 1;
// Hare's multi-type arm `case (inst_lit |
// inst_charset | inst_any)` (ha:407) is loud-rejected
// by design BOTH stages (PE5; Hare-parity task
// ww-core #13) — three void arms. The scrutinee binds
// through a typed let (the run_thread spelling).
let cur: inst = insts[term_start_idx];
match (cur) {
case inst_lit => void;
case inst_charset => void;
case inst_any => void;
case inst_groupend => {
// dead until the group fold ('(' is loud).
// Hare propagates with `?` (ha:410-411);
// `?` into compile's >32B tagged return is
// loud-stopped (#38b) — the explicit D13
// match, harec's own desugaring.
match (find_last_groupstart(
insts[0:term_start_idx])) {
case let e: error => return e;
case let sz: size => { term_start_idx = sz; };
};
};
case inst_groupstart =>
return "Unused '?'": error;
case =>
return "Misused '?'": error;
};
let after_idx: size = (insts.len: size) + 1;
insert(insts[term_start_idx], (after_idx: inst_split));
shift(insts[term_start_idx + 1:]);
};
case '*': { // regex.ha:421-443
if (r_idx == 0 || insts.len == 0) {
return "Unused '*'": error;
};
let new_inst_offset: size = 1;
let jump_idx: size = (insts.len: size) + new_inst_offset;
let after_idx: size = jump_idx + 1;
let term_start_idx: size = (insts.len: size) - 1;
let cur: inst = insts[term_start_idx];
match (cur) {
case inst_lit => void;
case inst_charset => void;
case inst_any => void;
case inst_groupend => {
// dead until the group fold; D13 match, cf '?'
match (find_last_groupstart(
insts[0:term_start_idx])) {
case let e: error => return e;
case let sz: size => { term_start_idx = sz; };
};
};
case inst_groupstart =>
return "Unused '*'": error;
case =>
return "Misused '*'": error;
};
let split_idx: size = term_start_idx;
term_start_idx += new_inst_offset;
insert(insts[split_idx], (after_idx: inst_split));
shift(insts[split_idx + 1:]);
append(insts, (split_idx: inst_jump));
};
case '+': { // regex.ha:444-459
if (r_idx == 0 || insts.len == 0) {
return "Unused '+'": error;
};
let term_start_idx: size = (insts.len: size) - 1;
let cur: inst = insts[term_start_idx];
match (cur) {
case inst_lit => void;
case inst_charset => void;
case inst_any => void;
case inst_groupend => {
// dead until the group fold; D13 match, cf '?'
match (find_last_groupstart(
insts[0:term_start_idx])) {
case let e: error => return e;
case let sz: size => { term_start_idx = sz; };
};
};
case inst_groupstart =>
return "Unused '+'": error;
case =>
return "Misused '+'": error;
};
append(insts, (term_start_idx: inst_split));
};
case '.': { // regex.ha:460-461
let av: inst_any;
let v: inst = av;
@@ -187,22 +418,36 @@ export fn compile(expr: str) (regex | error | nomem) = {
};
case ']': // regex.ha:315-316 — literal outside a bracket
append(insts, (r: inst_lit));
case '\\', '^', '$', '[', '(', ')', '|', '{', '?', '*', '+':
// fold-2a boundary: regex.ha:286-459 arms deferred.
case '[', '(', ')', '{':
// fold-3 boundary: the bracket (ha:313-334 + 268-275),
// group (317-334) and repetition (368-401) arms ride
// later folds.
return "regex: metacharacter not yet ported": error;
case: // regex.ha:462-463
append(insts, (r: inst_lit));
};
was_prev_rune_pipe = (r == '|'); // regex.ha:465
r_idx += 1;
};
// regex.ha:475-477. `$` appends true: inst_match — deferred, so
// the guard can only see no-match today; kept verbatim.
// handle whole expression alternation (regex.ha:470-473). Index
// loop + if+abort: the by-value range over an indexed element
// segfaults (#57) and the assert builtin is wwstage-broken (#58).
for (let k: size = 0; k < (jump_idxs[0].len: size); k += 1) {
let jump_idx: size = jump_idxs[0][k];
let cur: inst = insts[jump_idx];
if (!(cur is inst_jump)) {
abort("regex: alternation fixup: not a jump");
};
insts[jump_idx] = ((insts.len: size): inst_jump);
};
// regex.ha:475-477. `$` appends true: inst_match, so the guard
// sees real anchored programs now.
if (insts.len == 0 || !(insts[insts.len - 1] is inst_match)) {
append(insts, (false: inst_match));
};
// regex.ha:479-484. The alternation fixup (ha:470-473) drops with
// jump_idxs.
// regex.ha:479-484.
return regex {
insts = insts,
charsets = charsets,
@@ -277,8 +522,9 @@ fn add_thread(threads: *[]thread, parent_idx: size, new_pc: size) (void | nomem)
return;
};
// ref/hare/regex/regex.ha:589-742. Only the arms fold-2a's compile()
// can emit execute (skip + match non-consuming; lit + any consuming);
// ref/hare/regex/regex.ha:589-742. Only the arms compile() can emit
// execute (skip + split + jump + match non-consuming; lit + any
// consuming — split/jump went live with fold 3's `|`/`?`/`*` arms);
// every other inst arm is one loud not-yet-ported abort — the fold
// boundary, the compile() metachar discipline. The "unreachable"
// aborts on lit/any inside the non-consuming loop (ha:604-605) and
@@ -307,10 +553,13 @@ fn run_thread(
match (re.insts[(*threads)[i].pc]) {
case inst_lit => abort("regex: unreachable");
case inst_any => abort("regex: unreachable");
case inst_split =>
abort("regex: inst_split not yet ported");
case inst_jump =>
abort("regex: inst_jump not yet ported");
case let z: inst_split => { // regex.ha:606-609
let new_pc: size = (z: size);
add_thread(threads, i, new_pc)?;
(*threads)[i].pc += 1;
};
case let z: inst_jump => // regex.ha:610-611
(*threads)[i].pc = (z: size);
case inst_skip => {
let new_pc: size = (*threads)[i].pc + 1;
(*threads)[i].start_idx = str_idx;