ww source: reject malformed UTF-8

This commit is contained in:
2026-08-22 03:14:53 +09:00
parent d61805262e
commit 4069eda942
10 changed files with 1146 additions and 30 deletions

View File

@@ -459,6 +459,48 @@ remains a valid literal value. This rule is limited to raw U+0000; malformed
UTF-8 and the independent per-source BOM rule retain their existing, separate
contracts.
After filename and test-role eligibility and before package-clause
classification, every malformed UTF-8 byte in a selected production,
same-package test, external-package test, or test-only source is one positioned
`invalid UTF-8 encoding` error. The shared coordinator scans selected physical
files in existing byte-sorted order and reports all malformed bytes in the
first invalid file before package grouping, import discovery, or delegated
tools. The C and WW source decoders consume each malformed byte, omit it from
their logical streams, and resume, so the byte cannot manufacture or alter a
package token, import spelling, binding, or edge or produce stage-specific
fallback recovery. Correctly encoded U+FFFD and other valid non-ASCII text in
WW's permitted comment and literal contexts remain valid. A malformed
multi-byte spelling is diagnosed byte by byte according to width-one UTF-8
decodes. Raw NUL and per-source BOM diagnostics remain independent and
interleave with malformed-byte diagnostics in physical byte order.
Malformed selected source rejects before package/test variant construction,
generated main, compiler, assembler, archiver, linker, runtime, accounting,
retention, or publication. The existing attributable explicit test request
still owns its final `FAIL\n`; no package `ok` line or test-result cache is
created. Cold rejection leaves no unit, `.wwi`, assembly, object, archive,
binary, capture, status, stage, or transaction residue. Warm rejection commits
nothing and preserves the prior generation and public output byte for byte;
restoring the valid bytes follows ordinary exact-content reuse. Producer,
runtime, publication-only, and cleanup failures are unchanged because source
rejection precedes those phases. Validation is source/request-local, adds no
process or signal boundary, cannot contaminate an overlapping valid request,
and leaves interruption and owned-process cleanup with their established
owners. Wrong-target and ordinary-build-excluded test sources remain unread by
this semantic preflight.
`malformed_utf8_is_rejected_in_every_selected_source` in
`test/package/package_test.ww`, together with focused C and WW lexer coverage,
is the focused proof owner for valid encodings and encoded U+FFFD; stray leads
and continuations; overlong, surrogate, out-of-range, truncated, repeated, and
token-boundary cases; BOM/NUL interaction; root and dependency builds; source
versus import-resolution precedence; all test roles; wrong-target exclusion;
cold and warm rejection; exact restoration/reuse; parallel isolation;
diagnostic and valid-artifact stage parity; and absence of residue. Concrete
post-change measurements and hashes are recorded only after focused and full
validation. The completed BOM, raw-NUL, and exact output-option slices are not
reopened, and canonical dotted package/import identity remains unchanged.
After that eligibility boundary and the coordinator's required package-clause
classification and production `@test` validation parses, the delegated loader
performs selected-basename Go 1.26.5 simple-fold preflight before its graph