ww source: reject raw NUL characters
This commit is contained in:
@@ -8545,6 +8545,119 @@ and diagnostic text is not a persisted-byte contract. Build workdir format
|
||||
remains `18`, test workdir format remains `19`, and semantic storage format
|
||||
remains `3`.
|
||||
|
||||
### 11.43 Implemented selected-source U+0000 rejection
|
||||
|
||||
Every selected physical `.ww` source rejects a raw byte `00` (U+0000) at its
|
||||
physical line and byte column with exactly `invalid NUL character`. The rule is
|
||||
source-wide: comments, interpreted-string text, rune text, and between-token
|
||||
positions cannot turn a raw NUL into payload. An escape spelling such as
|
||||
`\x00` remains a legal literal value because it is not byte `00` in
|
||||
the source file. This section is only the raw-U+0000 rule: malformed UTF-8
|
||||
and the independently implemented per-source BOM boundary in §11.42 are not
|
||||
changed or broadened here.
|
||||
|
||||
#### Pinned evidence, applicability, and measured prior behavior
|
||||
|
||||
The sole authority is official Go 1.26.5 at
|
||||
`c19862e5f8415b4f24b189d065ed739517c548ba`. Its compiler owner,
|
||||
`(*source).nextch` in
|
||||
[`cmd/compile/internal/syntax/source.go`, lines 113–165](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/syntax/source.go#L113-L165),
|
||||
detects ASCII zero at lines 121–129, reports `invalid NUL character`, and
|
||||
continues decoding. `TestScanErrors` pins the positioned diagnostic at
|
||||
[`cmd/compile/internal/syntax/scanner_test.go`, lines 587–600](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/syntax/scanner_test.go#L587-L600),
|
||||
and compiler testdata [`test/nul1.go`, lines 7–52](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/test/nul1.go#L7-L52)
|
||||
requires NUL errors in strings, raw strings, line/block comments, and ordinary
|
||||
source. Go's independent public scanner has the same rule in
|
||||
[`go/scanner/scanner.go`, lines 63–108](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/scanner/scanner.go#L63-L108)
|
||||
and its tests at lines 790–819. These are **behavior directly implemented or
|
||||
asserted by pinned Go**.
|
||||
|
||||
Before this change, directly measured Cstage and WWstage package builds both
|
||||
accepted a NUL in a line comment, wrote it into the committed synthetic unit,
|
||||
published byte-identical archive/interface products, and allowed a dotted local
|
||||
importer to link and run. The retained audit commands used `out/bin/ww` and
|
||||
`out/bin/ww_ww` with `WW_SRCLIB=/home/kimchi/src/ww/lib` over
|
||||
`/tmp/ww-pkgaudit.qtjcpO/src/nulcomment`; both exited zero with empty streams.
|
||||
The same acceptance applied to raw NUL in strings and runes. Those observations
|
||||
are **directly measured WW behavior**. Applying pinned Go's physical-source
|
||||
rule independently to WW's selected files is **behavior derived from the
|
||||
pinned implementation** and is applicable without importing Go's module,
|
||||
manifest, registry, lock, cache, database, CAS, network, generalized-import,
|
||||
or source-level build-expression model.
|
||||
|
||||
#### Ownership, timing, and four axes
|
||||
|
||||
The direct-frontend semantic owners are the logical source-decoder helpers in
|
||||
`cmd/wcc/lex.c` and their exact twins in `lib/ww/syntax/lex.ww`: each raw zero
|
||||
is consumed, reports the same positioned error, and is omitted before token
|
||||
recovery. Filtered token spans preserve that decoder rule through identifiers,
|
||||
numbers and suffixes, directives, escapes, operators, comments, and EOF. The
|
||||
line-comment decoder consumes each body once before classifying an internal
|
||||
module directive, so NUL filtering does not make generated path handling
|
||||
superlinear. The
|
||||
shared `internal/wwpackage/package.ww` coordinator owns public package/test
|
||||
diagnostic precedence: its length-aware preflight reports every raw NUL in an
|
||||
invalid physical source before its manual package-clause classifier. It is not
|
||||
another identity policy. Driver source slurping and synthetic-unit composition
|
||||
preserve byte lengths and are not semantic owners.
|
||||
|
||||
- **Go-like build:** after fixed-target filename selection, an invalid selected
|
||||
root or dependency rejects during loading, before graph completion and before
|
||||
compiler, assembler, archiver, linker, install, publication, or execution.
|
||||
Source rejection in a root precedes resolution of that root's missing
|
||||
imports. A wrong-target file is excluded before this rule and remains unread
|
||||
by its semantic owners.
|
||||
- **Go-like test:** selected production, same-package, external-package, and
|
||||
test-only source each receive the rule before variant construction. Failure
|
||||
emits the established attributable `FAIL\n` without a generated main, test
|
||||
process, accounting, `ok` result, retained binary, or public test product.
|
||||
- **Go-like package:** each selected physical file owns its diagnostic and
|
||||
position. Declared package name, source role, package conflict handling,
|
||||
command/test family, physical directory, and exact canonical dotted identity
|
||||
are unchanged.
|
||||
- **Go-like import:** invalid bytes create no import edge or graph node. Valid
|
||||
import spelling, aliases, local/vendor/internal resolution, visibility,
|
||||
cycle handling, and initialization order remain unchanged.
|
||||
|
||||
Thus raw NUL is never an input to manifest-free package identity, graph/action
|
||||
keys, symbols, `.wwi`, archive naming, publication names, or persistence keys.
|
||||
It neither changes local dotted-import boundaries nor introduces a manifest.
|
||||
|
||||
#### Failure, publication, persistence, and parity
|
||||
|
||||
Cold invalid requests create no unit, assembly, object, archive, executable,
|
||||
capture, `.new`, `.install`, `.wwtxn.*`, or public output. A warm edit that
|
||||
introduces NUL stops before a producer or install action, preserving the prior
|
||||
committed unit/interface/assembly/object/archive generation, tool vouchers,
|
||||
stamp, and public output byte for byte. Removing the NUL restores the ordinary
|
||||
selected-source fingerprint; exact restoration may reuse the earlier generation.
|
||||
Existing producer/runtime failure and transaction rollback remain their own
|
||||
owners because this branch creates no new rollback mechanism.
|
||||
|
||||
Lexer and coordinator state are request/source-local. Concurrent valid and
|
||||
invalid requests keep independent workdirs, captures, diagnostics, and
|
||||
products; an invalid request cannot contaminate a valid sibling. The rule adds
|
||||
no process, wait, or cancellation boundary, so signal, timeout, interruption,
|
||||
process-group cleanup, and ordinary scratch cleanup retain their established
|
||||
owners. Validation itself leaves no durable residue. Cstage and WWstage are
|
||||
semantic twins: diagnostics match exactly, and valid unit/compiler/product
|
||||
bytes retain their existing byte-identity contract.
|
||||
|
||||
`raw_nul_is_rejected_in_every_selected_source` in
|
||||
`test/package/package_test.ww`, with focused C and WW lexer coverage, proves
|
||||
literal and comment contexts plus adjacency recovery across escapes,
|
||||
identifiers, numbers and typed suffixes, operators, comment delimiters, and
|
||||
EOF; it also proves selected/imported builds, wrong-target exclusion, every
|
||||
directory-test source role, precedence, cold cleanup, warm rollback and reuse,
|
||||
concurrent isolation, valid escaped-NUL behavior, runtime/publication behavior,
|
||||
and no-residue/parity observations.
|
||||
|
||||
No format bump. This changes invalid-source acceptance and diagnostics only;
|
||||
valid source composition and valid `.wwi`, assembly, object, archive,
|
||||
executable, and retained-test-product bytes are unchanged. Build workdir format
|
||||
remains `18`, test workdir format remains `19`, and semantic storage format
|
||||
remains `3`.
|
||||
|
||||
## 12. Candidate architectures and hard-gate decision
|
||||
|
||||
Five candidates were developed as coherent systems, not as feature bins.
|
||||
|
||||
10
docs/spec.md
10
docs/spec.md
@@ -60,6 +60,16 @@ every other source position, including inside string and rune literals and
|
||||
comments. Apart from that marker rule, the lexer operates on bytes and
|
||||
non-ASCII bytes are legal only inside string and rune literals and comments.
|
||||
|
||||
Each raw byte `00` (U+0000) is invalid at every physical source position,
|
||||
including in comments and string or rune literal text. It produces one
|
||||
positioned `invalid NUL character` error at that byte's source position and
|
||||
is omitted from the lexer's logical character stream before token recovery.
|
||||
It therefore cannot split an identifier, number, operator, escape, or comment
|
||||
boundary into different tokens. This is a source-representation rule, before
|
||||
package/import interpretation; it does not make U+0000 a package, import, or
|
||||
artifact identity component. An escape spelling such as `"\\x00"` is not a
|
||||
raw source byte and remains a legal literal value.
|
||||
|
||||
### 2.2 Comments
|
||||
|
||||
Line comments only, introduced by `//` and running to end of line. There
|
||||
|
||||
@@ -429,6 +429,26 @@ before either operation. The marker is source representation only and never
|
||||
package/import/action/artifact/publication/persistence identity; no test result
|
||||
is cached.
|
||||
|
||||
After filename eligibility and before package-clause classification, each raw
|
||||
source byte `00` in a selected production, same-package test,
|
||||
external-package test, or test-only source is one positioned
|
||||
`invalid NUL character` error. Comments and literal text do not hide it. The
|
||||
shared coordinator owns that early package/test rejection and reports every
|
||||
raw NUL in the invalid physical source. On complete direct
|
||||
frontend inputs, the C and WW source decoders diagnose each raw NUL and omit it
|
||||
from their logical character streams before token recovery. Thus a NUL cannot
|
||||
split an escape, identifier, number or suffix, operator, comment delimiter, or
|
||||
EOF boundary into a stage-dependent second error. Test loading maintains a
|
||||
stage-equal source-diagnostic stream before import discovery, graph actions,
|
||||
producers, test execution, accounting, result output, or retained publication.
|
||||
An invalid selected test request emits only the existing attributable
|
||||
`FAIL\n`; it does not construct a variant, generated main, test child, or new
|
||||
persistent generation. Wrong-target exclusion remains first, so an excluded
|
||||
file with a raw NUL has no diagnostic or persistence effect. A `\\x00` escape
|
||||
remains a valid literal value. This rule is limited to raw U+0000; malformed
|
||||
UTF-8 and the independent per-source BOM rule retain their existing, separate
|
||||
contracts.
|
||||
|
||||
After that eligibility boundary and the coordinator's required package-clause
|
||||
classification and production `@test` validation parses, the delegated loader
|
||||
performs selected-basename Go 1.26.5 simple-fold preflight before its graph
|
||||
|
||||
Reference in New Issue
Block a user