test: prove recursive local package patterns

docs/build-system.md

@@ -2806,8 +2806,11 @@ imports-only parser, unions duplicate imports, byte-sorts direct edges, interns
 canonical directory actions, and reports self-imports and stable cycle chains
 before compilation.
 
-A directory package consists of its immediate regular non-symlink `.ww` files,
+A directory package consists of its immediate `.ww` entries whose basenames do
-excluding `*_test.ww`, in byte-sorted filename order. Every selected file must
+not begin `.` or `_`: regular files and symlinks targeting regular files are
+included under the entry name, while symlinks targeting directories are
+ignored. The production variant excludes `*_test.ww`; each variant retains
+byte-sorted filename order. Every selected file must
 declare the same package. An ordinary importable directory's declaration must
 equal the final component of its import path. A selected command directory
 instead declares `package main` to validate command kind while retaining its
@@ -2925,11 +2928,13 @@ importer under the existing propagation rule.
 ### 11.7 Implemented directory package-test slice
 
 Directory tests now enter that same local package loader and build path. The
-supported manifest-free commands are `ww test DIR`, `ww test DIR/...`, and
+supported manifest-free forms include `ww build DIR...`, `ww build DIR/...`,
-their existing `-run`, `-filter`, `-list`, `-timeout-ms`, `-j`, `-c`, and `-w`
+`ww test DIR...`, and `ww test DIR/...`, with overlapping direct and recursive
-forms. `ww test -c -o test.bin DIR` names the result when the selected directory
+roots. Test retains its existing `-run`, `-filter`, `-list`, `-timeout-ms`,
-has one test variant; the coordinator rejects one output name for a multi-variant
+`-j`, `-c`, and `-w` forms. `ww test -c -o test.bin DIR` names the result when
-or recursive request. Explicit `ww test FILE` retains its compatibility path.
+the selected directory has one test variant; the coordinator rejects one output
+name for a multi-variant or recursive test request. Explicit `ww test FILE`
+retains its compatibility path.
 
 The test coordinator still discovers requested directories, enumerates the
 test package names, selects variants, executes independent binaries, and emits
@@ -2978,9 +2983,8 @@ invocation, each still-unbound directory is finalized by this exact algorithm:
 1. For every import-resolution context that reached the directory, walk that
    context's roots in its normal forward precedence: the selected package's
    directory, explicit `-I` roots in command order, then `WW_SRCLIB` or the
-   selected toolchain source root. Recursive package-test requests privately
+   selected toolchain source root. A request pattern never becomes an import
-   insert their symlink-resolved discovery root before user `-I` roots, so
+   root and therefore cannot shorten, replace, or donate package identity.
-   descendants retain their complete relative identity.
 2. Canonicalize each candidate root and require the package directory to be a
    strict descendant. Every relative path component must be a non-keyword WW
    identifier. Convert separators to dots, then resolve that relative spelling
@@ -3019,11 +3023,13 @@ its derived `_test` compiler path.
 
 The derivation and diagnostics are implemented symmetrically in
 `cmd/ww/main.c` and `selfhost/cmd/ww/main.ww`. The package coordinator in
-`internal/wwpackage/package.ww` supplies the canonical recursive discovery root,
+`internal/wwpackage/package.ww` supplies canonical selected directories and
-preserves an explicitly resolved logical request identity, and keys a persistent
+variant descriptors, preserves an explicitly resolved identity only for one
-request workdir only by the canonical discovery directory. `w6c` and `wcc`
+direct request, and forwards a caller's `-w` semantic-action store unchanged.
-continue to consume and validate the finalized dotted identity; neither tool
+It never derives identity or persistent layout from a pattern traversal prefix
-performs directory lookup or introduces a package registry.
+and does not add that prefix to import search. `w6c` and `wcc` continue to
+consume and validate the finalized dotted identity; neither tool performs
+directory lookup or introduces a package registry.
 
 Artifact publication follows the semantic action instead of product order:
 production uses the full finalized ordinary identity, internal appends
@@ -3368,7 +3374,7 @@ discovery each compile the shared production action once; recursive `a/foo`
 and `b/foo` directories declaring the same `package foo` publish distinct
 `a.foo` and `b.foo` variants; and two outside-root `package foo` directories
 coexist in one command under distinct reversible local identities. Equivalent
-recursive spellings reuse the same persistent request directory without new
+recursive spellings reuse the same persistent semantic-action store without new
 compilation, while source imports of the reserved local namespace reject before
 tool invocation. The exact-argv command root declares `package main` but keeps
 its non-`main` canonical identity in the unit, export, archive, compiler argv,
@@ -3594,23 +3600,18 @@ existing complete-path owner check before tools. This preserves collision
 checking without serializing machine-specific canonical directories into units
 or exports and without a sidecar, registry, or new metadata protocol.
 
-The package coordinator similarly preserves the reversible existing
+The package coordinator derives no persistent container from a request
-`d_<escaped-canonical-request-directory>` component while it fits.  A longer
+directory or pattern. `-w DIR` names the driver's semantic-action store itself,
-canonical request directory uses:
+and the driver independently derives every action locator in that store from
-
+the full semantic tuple above and validates each committed semantic owner.
-```text
+Equivalent direct, logical, relative, absolute, dotted, recursive, duplicate,
-d_<lowercase-sha256("ww-request-workdir-v1:" || canonical-directory)>
+and explicit-root-symlink requests that select the same canonical actions can
-```
+therefore reuse the same slots; request shape cannot split or alias persistent
-
+state. For a delegated request only, a missing caller `-w` directory is created
-This key selects only the driver workdir container; it is never a package
+after graph, identity, visibility, cycle, closure, and output preflight. An
-identity or action address.  Equivalent logical, relative, absolute, `.` and
+explicit root symlink is followed and canonicalized; a source entry symlink to
-explicit root-symlink requests canonicalize before this derivation.  Inside the
+a regular file is followed under the entry name, a source-shaped symlink to a
-container the driver independently derives every action locator from its full
+directory is ignored, and symlinked recursive children are not traversed.
-semantic tuple and validates the committed semantic owner, so even a
-coordinator-key collision cannot silently reuse an unrelated package action or
-invoke a tool with a shortened identity.  Explicit root symlinks are followed
-and canonicalized; source-file symlinks and symlinked recursive children retain
-their existing rejection/skipping rules.
 
 The old `SEP_IMPORT_PATH_MAX` and all corresponding 255-byte WWstage import,
 variant, local-identity, and generated-main checks are removed.  Package names,
@@ -3778,7 +3779,7 @@ actions and its exact linker line still contains the root followed by the full
 `p000` through `p299` transitive archive chain and runtime archive, proving that
 closure construction—not the wide root's direct imports—crosses the old boundary.
 The observer compares every unit, export, assembly, object, archive, and binary
-across stages. A second equivalent persistent request invokes no compiler or
+across stages. A second equivalent request against the same persistent store invokes no compiler or
 assembler. Changing `p257`'s export recompiles exactly `p257`, direct importer
 `p256`, and the wide root, and stops before `p255` after `p256` regenerates a
 byte-identical export. Closing the chain at `p299 -> p000` produces the complete
@@ -3801,7 +3802,7 @@ The same observer then exercises the public `ww test <tree>/...` coordinator
 path. The generated tree has 257 real directories: the 52 test-bearing
 directories above plus 205 production-only directories. The coordinator forms
 and passes 309 products and 258 contexts (including support) in one driver
-request. Reusing the direct request's persistent workdir preserves all 270
+request. Reusing the direct request's persistent semantic-action store preserves all 270
 existing actions and adds exactly 410 production/generated-main actions, for 680
 distinct actions. Both stages run every product, produce byte-identical ordered
 coordinator output and all 680 action artifacts, and perform no compilation on a
@@ -3810,8 +3811,8 @@ and all 205 no-test result labels, while a builder-boundary observer records
 exactly one invocation containing all 309 descriptors for each cold and warm
 public request. Existing focused observers continue to prove
 dependency-first/root-first canonical reuse, root-only/combined artifact
-identity, exact reordered-product trace bytes, and persistent request-directory
+identity, exact reordered-product trace bytes, and request-shape-independent
-stability.
+persistent action reuse.
 
 This representation follows the semantic separation and scalable action
 construction in the pinned official Go 1.26.5 source, identified by
@@ -4141,9 +4142,10 @@ Generated-main-to-variant, generated-main-to-support, and coordinator product
 edges remain synthetic and receive no map or retroactive source legality.
 `cmd/wwtest` remains a dispatcher. `internal/wwpackage` still discovers and
 classifies source groups and submits one command-global union; it performs no
-vendor resolution. Its only related state change is transactional cleanup: if
+vendor resolution. It creates only its removable command-temporary coordination
-it created a cold hashed request workdir and the driver rejects before a tool
+tree. The private driver creates a missing delegated `-w` directory and a
-stamp exists, it removes that empty request directory.
+requested output directory only after all semantic graph and output preflight
+succeeds; a rejected request therefore leaves neither directory behind.
 
 Resolution, identity collision checks, contextual legality, dependency-failure
 propagation, cycles, command kind, publication paths, and action closures all
@@ -4208,6 +4210,253 @@ direct exports/import maps, archive-only link closures, runtime output,
 stage-equal normalized argv/artifacts/binaries, warm no-op production, and cold
 and warm rejection-state preservation.
 
+### 11.17 Implemented manifest-free recursive local package-pattern selection
+
+Build and test now share one local request selector. A positional spelling with
+no `...` is one explicit directory root. In a spelling containing `...`, each
+occurrence in a valid UTF-8 spelling has Go's regular-expression wildcard
+semantics; an invalid UTF-8 pattern matches nothing, and a final `/...` also
+matches the directory before that suffix. WW applies the local matcher to its
+existing manifest-free `DIR/...` interface as well as `./...`, `../...`, and
+absolute spellings. It does not interpret the non-filesystem portion as a
+module or registry path. Multiple direct roots and patterns may be mixed in one
+command.
+For build, `--` before the first positional ends option parsing and every
+following argument is a package selector, including a spelling that begins
+with `-`. Once the first positional has already ended flag parsing, a later
+`--` is itself another package argument.
+
+Pattern expansion is request processing only. For each pattern, traversal starts
+at the directory prefix before the first `...` and is bounded to that physical
+tree. The selector never obtains a traversal or source-root boundary from
+another product. It produces canonical physical directory roots; only imports
+parsed from their real source files add dependency edges. The request spelling,
+wildcard prefix, match membership, output name, request order, and discovery
+order never become a package import identity or action key and never enter an
+import search path.
+
+The recursive eligibility rules are:
+
+- Directory entries are read completely and byte-sorted before processing.
+  Every recursively encountered directory whose basename begins `.` or `_`, or
+  equals `testdata`, is pruned with its subtree. An explicit literal selection
+  bypasses these traversal exclusions, so those directories remain legal direct
+  roots.
+- A directory owns only its immediate `.ww` directory entries. Basenames
+  beginning `.` or `_` are ignored. As in `go/build`, a source symlink whose
+  target is a regular file is read under the symlink entry's byte-sorted name;
+  a source-shaped symlink to a directory is ignored. A subdirectory never
+  donates sources to its parent.
+- A directory with at least one production or `*_test.ww` source is eligible.
+  A recursively encountered source-empty directory is silently skipped. A
+  direct source-empty root is an error. A malformed source-bearing directory is
+  retained as a root and fails during ordinary package-clause or driver loading;
+  malformed files in excluded or source-empty trees do not poison the request.
+- The traversal does not prune `vendor`. Instead a wildcard cannot consume a
+  non-terminal exact path component named `vendor`. Thus `DIR/...` may select a
+  code-bearing terminal `DIR/vendor` but not `DIR/vendor/x`.
+  `DIR/vendor/...` explicitly selects that vendor root and its descendants until
+  another non-terminal `vendor` becomes a barrier. `vendorx` is ordinary.
+
+The explicit traversal root is opened after following a directory symlink, as
+in Go. It may therefore name a target outside the lexical spelling, but the
+target becomes the canonical traversal boundary. Directory symlinks encountered
+below that root are never followed, so they cannot escape, create cycles, or
+change selection. A cyclic explicit root is rejected while canonicalizing it.
+Relative, absolute, dotted, and explicit-root-symlink spellings that reach one
+physical package collapse to one canonical root. This physical interning is
+WW's stronger command-global identity rule; it deliberately avoids Go's few
+GOPATH cases in which different lexical import paths can retain distinct package
+objects.
+
+Raw requested spellings are byte-sorted first and each is then lexically cleaned
+before traversal. Matched source paths are canonicalized, sorted by canonical
+directory and filename, and deduplicated. Products are then byte-sorted by
+canonical directory and variant. Duplicate patterns, overlapping patterns, and
+canonical aliases therefore select one root/product and reuse one action. Go
+itself processes patterns in argv order and suppresses later package objects; WW
+performs the stronger final canonical sort required by its
+request-order-independent command universe. Reversing request or product order
+does not change roots, diagnostics, normalized tool arguments, artifacts, or
+runtime output.
+
+An unmatched pattern emits:
+
+```text
+ww: warning: "PATTERN" matched no packages
+```
+
+`PATTERN` is quoted with the pinned `strconv.Quote` rules, including
+deterministic escapes for quotes, backslashes, controls, non-printing Unicode,
+and malformed UTF-8 bytes. Warnings are emitted in the
+sorted request order. With no remaining roots,
+ordinary build without `-o` succeeds as an empty build, while test reports
+`ww test: no packages to test`. A build with a non-directory `-o` reports no
+packages to build; a directory `-o` reports no main packages to build. A
+non-directory `-o` still requires exactly one production root. An existing
+directory or spelling ending in `/` receives each selected command under its
+canonical directory basename; non-main selected roots receive no named output.
+Two commands with the same destination basename are rejected rather than
+overwriting one another. All selection, canonicalization, package-clause,
+duplicate-output, and unusable-request diagnostics precede producer execution.
+Raw `-o` and `-w` spellings and every derived directory-fan-out output, cold
+scratch name, persistent tool-record name, and package artifact are bounded and
+validated symmetrically before tools; a raw spelling that fits but whose suffix
+or command basename does not fit is rejected with the same Cstage/WWstage
+diagnostic and no filesystem publication.
+For a delegated multi-root or recursive `-S` build, `-w` is required so the
+assembly outputs have caller-owned persistent destinations instead of vanishing
+with the coordinator's temporary plan.
+
+Build and test begin with the same eligible canonical directory set. Recursive
+build removes a source-bearing root that has only test files; an explicitly
+selected test-only directory remains an unusable build root and fails. Build
+creates one production product per remaining directory. Test retains test-only
+directories and, after selection, constructs the already specified isolated
+production/no-test, internal production-plus-test, external `_test`, support,
+and generated-main actions. Pattern expansion does not create those variants
+and cannot make one variant visible to another.
+
+Vendor selection remains distinct from vendor import resolution. Selecting a
+directory below `vendor`, literally or through an explicitly vendor-rooted
+pattern, keeps its complete canonical local identity; it is never shortened to
+the suffix after `vendor`. A generic recursive pattern does not expose vendored
+descendants as ordinary short command roots. Independently, an allowed real
+source import still searches nearest-first below local `vendor`, creates the
+expanded identity described in section 11.16, supplies the required
+`--import-map`, and performs spelling and visibility checks for that importer
+even when the canonical action already exists.
+
+The ownership split is exact:
+
+- `cmd/ww/main.c` and `selfhost/cmd/ww/main.ww` recognize recursive and
+  multi-root build/test requests symmetrically and delegate them. Their private
+  build route consumes production product descriptors in one command-global
+  package universe. It records library-root completion without linking and
+  links each command product independently. Recursive build forwards `-S`,
+  `-L`, and `-l` through the same private product route; assembly-only products
+  receive completion markers only after their producer pass succeeds. Both
+  stages allocate the same bounded delegation argv and inherit the existing
+  environment directly, so delegation adds no WWstage-only environment-copy
+  allocation or failure point.
+- `internal/wwpackage/package.ww` owns pattern cleaning and matching, bounded
+  directory traversal, source-bearing eligibility, canonical root/product
+  sorting and deduplication, build-versus-test product classification, output
+  coordination, and deterministic reporting. It does not resolve a source
+  import and does not prepend a pattern root to `-I`.
+- `cmd/wwtest` remains a dispatcher. The Cstage and WWstage drivers own package
+  enumeration, canonical identity, contextual import resolution, graph loading,
+  variants, compilation, archive construction, linking, publication, and
+  persistent reuse.
+
+Consequently the existing action and tool contracts remain unchanged after
+root selection. Every action unit contains only the owner's byte-sorted source
+files. A compiler receives exactly the sorted, deduplicated `.wwi` exports of
+direct source dependencies and any required vendor import-map binding. Each
+executable link receives its root archive and complete reachable archive
+closure, never a `.wwi`. Canonical duplicate roots reuse the same action; the
+pattern text and the product receiving an output do not affect semantic or
+persistent identity.
+
+Selection and package-clause validation finish before the coordinator creates
+its removable temporary plan, and the coordinator never creates persistent
+state. Driver graph, identity, visibility, cycle, closure, and output validation
+finish before a missing delegated work or output directory is created and
+before scratch, voucher, status, tool-state, publication, or producer mutation.
+If later setup of another requested directory fails, both drivers remove every
+empty path prefix created by that setup while preserving all pre-existing caller
+directories. A cold rejected pattern leaves no partial request state. The same
+rejection against an existing caller work root leaves its marker and every
+committed artifact, voucher, and tool record byte-unchanged and invokes no
+producer. A valid equivalent direct or recursive warm request remains a
+package-production no-op; changed exports continue to propagate through direct
+dependencies only.
+
+This slice follows only official Go 1.26.5 source at commit
+`c19862e5f8415b4f24b189d065ed739517c548ba`:
+
+- local literal versus wildcard handling, the prefix before the first `...`,
+  explicit root-symlink following, recursive exclusions, source-empty omission,
+  and malformed-directory retention are in
+  [`cmd/go/internal/search/search.go`, lines 276–418](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/search/search.go#L276-L418),
+  and its quoted unmatched-pattern warning is at
+  [lines 424–429](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/search/search.go#L424-L429);
+- wildcard syntax, the empty match for trailing `/...`, and the non-terminal
+  `vendor` barrier are in
+  [`cmd/internal/pkgpattern/pkgpattern.go`, lines 32–106](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/internal/pkgpattern/pkgpattern.go#L32-L106),
+  including invalid-UTF-8 rejection at lines 75–76,
+  with non-terminal vendor-element replacement implemented at
+  [lines 125–137](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/internal/pkgpattern/pkgpattern.go#L125-L137);
+- manifest-free request expansion, canonical package loading, first-occurrence
+  deduplication, and pattern membership are in
+  [`cmd/go/internal/load/pkg.go`, lines 2922–2965](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L2922-L2965),
+  while source imports alone recurse through package loading at
+  [lines 2024–2047](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L2024-L2047);
+- build uses that common matcher, implements single versus directory `-o`, and
+  omits wildcard-selected test-only roots at
+  [`cmd/go/internal/work/build.go`, lines 459–559](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L459-L559)
+  and
+  [lines 731–745](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L731-L745);
+- test uses the same package request set, reports an empty set, and constructs
+  isolated test variants only afterward at
+  [`cmd/go/internal/test/test.go`, lines 684–719](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L684-L719)
+  and
+  [lines 1133–1226](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L1133-L1226);
+- immediate directory ownership and hidden/underscore source-name exclusion
+  are in
+  [`go/build/build.go`, lines 859–914](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L859-L914)
+  (including regular-file source symlink following and symlink-to-directory
+  omission at lines 886–900),
+  parse-error retention and production/test classification are in
+  [lines 931–1036](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L931-L1036),
+  and `NoGoError` eligibility is in
+  [lines 1076–1082](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L1076-L1082); and
+- child symlinks are skipped by `Lstat`-based directory walking and entries are
+  traversed in byte order at
+  [`cmd/go/internal/fsys/walk.go`, lines 14–59](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/fsys/walk.go#L14-L59)
+  and
+  [`os/dir.go`, lines 109–125](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/os/dir.go#L109-L125).
+
+Build subcommand option termination is delegated by the Go command at
+[`cmd/go/main.go`, lines 312–321](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/main.go#L312-L321),
+with `--` termination implemented by
+[`flag/flag.go`, lines 1074–1089](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/flag/flag.go#L1074-L1089),
+and the parse loop stops at the first positional at
+[lines 1153–1176](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/flag/flag.go#L1153-L1176).
+
+Unmatched-pattern quoting uses the pinned `strconv.Quote` decoder and escape
+rules in
+[`strconv/quote.go`, lines 28–123](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/strconv/quote.go#L28-L123),
+including the exported `Quote` entry at lines 117–123, and the `IsPrint`
+algorithm at
+[`strconv/quote.go`, lines 515–559](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/strconv/quote.go#L515-L559),
+with its generated tables in
+[`strconv/isprint.go`, lines 8–733](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/strconv/isprint.go#L8-L733).
+
+No module cutoff, `go.mod`, module vendor mode, `vendor/modules.txt`, importcfg,
+build ID, registry, database, CAS, or network behavior is copied.
+
+The native observer `recursive_tree_discovery` generates every tree
+temporarily and runs Cstage and WWstage from independent cold roots. It covers
+ordinary and test-only roots, hidden/underscore/testdata exclusions and direct
+exceptions, terminal and explicit vendor patterns, an imported expanded vendor
+dependency at both package-local and ancestor vendor directories,
+overlapping/reversed/duplicate patterns independently in each stage, root,
+child-directory, regular-file, and directory-target source symlinks,
+canonical aliases, source-empty and malformed directories, build/test variant
+selection with an asserted common production-root set, middle-position and
+valid-UTF-8 wildcard edge cases, byte-sorted quoted unmatched diagnostics,
+multi-command directory output and duplicate-destination rejection, recursive
+assembly/link-flag forwarding, build `--` termination, raw-versus-derived
+output/scratch/work-path boundaries with zero producer calls, owner-only units,
+exact direct exports and import maps, archive-only links, normalized stage-equal tool argv and
+artifacts/binaries/output, warm no-op package production, and cold/warm pre-tool
+rejection-state preservation. The existing
+`vendor_directory_import_resolution` observer supplies the exact compiler
+`--import-map` argv proof for imported vendor dependencies, and the existing
+diamond and long-closure observers independently prove sorted/deduplicated
+direct `.wwi` cardinality and archive-only reachable link closures.
+
 ## 12. Candidate architectures and hard-gate decision
 
 Five candidates were developed as coherent systems, not as feature bins.