diff --git a/docs/build-system.md b/docs/build-system.md index 8e324d2f..1398d28c 100644 --- a/docs/build-system.md +++ b/docs/build-system.md @@ -2673,18 +2673,20 @@ not a self-contained functional toolchain outside the build tree. ### 11.3 Scaling, invalidation, and hidden inputs -The driver stores fixed `deps[256]` arrays and performs linear graph lookup. -Every package unit reads one interface per transitive dependency, so total -interface reads and copied interface text are quadratic on deep/dense graphs even -on a warm build. Package compiler/assembler work within one driver is serial; -Make gains parallelism only by launching independent top-level driver builds. +The driver performs deterministic linear action interning but now grows every +package dependency vector dynamically (section 11.14). Every package compiler +reads exactly one interface per direct dependency; transitive dependencies enter +only the executable archive closure. Package compiler/assembler work within one +driver remains serial; Make gains parallelism only by launching independent +top-level driver builds. The observed invalidation rules are: - a private change in a directory dependency rebuilds that package and the unconditional final link, but not importers; -- an exported change changes its `.wwi` and rebuilds the whole reverse-transitive - ancestor cone, even when an intermediate package's own interface is unchanged; +- an exported change rebuilds direct importers and continues through an ancestor + only while the regenerated direct-dependency export bytes change, stopping at + the first byte-identical regenerated interface; - a private change in a folded file import rebuilds its entire owner; - a link-only option reruns the always-executed link but not package compiles; - changing copied compiler or assembler bytes rebuilds every package; and @@ -3554,9 +3556,10 @@ allocations. Cstage's assembler no longer copies a line, operand, `TEXT`, or `DATA` symbol through 256-byte arrays, and the C checker no longer resolves a qualified type through a 128-byte prefix buffer. `PATH_MAX` remains only at actual host pathname and syscall boundaries; the 255-byte constant remains -only as the conservative internal basename component bound. `SEP_MAXPKG`, -`SEP_MAXPRODUCT`, and `SEP_MAXCONTEXT` remain action-count limits, not byte -limits on semantic identity. +only as the conservative internal basename component bound. The former +`SEP_MAXPKG`, `SEP_MAXPRODUCT`, and `SEP_MAXCONTEXT` action-count limits are +removed by the dynamically sized package-universe implementation in the next +section. The existing native observers exercise the new boundary with ordinary dotted identities over 255 bytes and punctuation-heavy reversible local identities @@ -3624,6 +3627,190 @@ build IDs, importcfg, module machinery, or scheduler: [`test.go`, lines 175–226](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L175-L226), [`test.go`, lines 228–293](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L228-L293)). +### 11.14 Implemented dynamically sized command-global package universe + +Package, dependency, resolution-context, selected-product, traversal, support, +order, and closure storage no longer has an arbitrary 256-element boundary. +This is a storage correction, not a new build abstraction: source imports still +form one command-global canonical package graph; each semantic package variant +still has one action; each compiler still receives exactly its direct exports; +and each executable linker still receives its complete reachable archive +closure. + +The Cstage representation is exact and deliberately small: + +- `sepgraph.pkg` is a dynamically allocated `struct seppkg *` with logical + count `n` and capacity `pkgcap`; +- `sepgraph.context` is a dynamically allocated `struct sepcontext *` with + logical count `ncontext` and capacity `contextcap`; +- each `seppkg.deps` is a dynamically allocated `int *` with `ndeps` and + `depcap`; +- each `seppkg.context_state` is a lazily extended, zero-filled + `unsigned char *` with `context_cap`; +- parsed `sepproduct` values are a dynamically allocated vector, and each + product stores its support-action index directly; and +- package-load frames and topological-DFS frames are temporary dynamic vectors, + replacing recursion proportional to graph depth. + +The WWstage representation is isomorphic. `sepgraph.pkg: []seppkg` and +`sepgraph.context: []sepcontext` use allocated slice length as capacity and keep +separate `n`/`ncontext` logical counts. Every `seppkg` owns a dynamically grown +`deps: []i32` with `ndeps` and a lazily zero-extended `contextstate: []u8`. +Products, load frames, and topological frames use typed dynamically allocated +slices. `internal/wwpackage` continues to construct one union command for all +selected directory-test groups, but now checks the complete +`12 + 6*products + 2*includes` builder argument count before allocating or +starting the driver. + +All graph and product growth starts at capacity 8 and doubles until it covers +the requested element count. Cstage clamps before `INT_MAX`, checks the element +count against `SIZE_MAX / sizeof(element)`, and publishes a `realloc` result only +after success. WWstage checks against the same signed 32-bit count boundary, +allocates a replacement typed slice, copies the live prefix, and publishes it +only after success. Context-state growth copies old bytes and explicitly zeros +the new tail. The shared deterministic failures are `ww: package graph is too +large` for an unrepresentable count and `ww: out of memory` for failed storage; +compiler and linker argument-count arithmetic is checked before allocation and +before any affected tool invocation. Each driver records allocation/size +failure during graph discovery and propagates it as a command-fatal load +result, rather than treating it as one product's semantic failure and starting +tools for a sibling root. The coordinator uses fallible dynamic storage for +discovered paths, source/folder/group/plan vectors, process handles, tool +environments, and complete builder/run argument vectors; it reports an +oversized product set or allocation failure before the corresponding +`exec.start` and cleans an already-created request temporary tree. Host pathname, +filesystem-component, process-argument, and available-memory boundaries remain +real host constraints; none is used as a disguised package-count maximum. + +Vector growth never changes semantic references. Dependency edges, resolution +contexts, selected-product roots and variant roots, generated-main/support +edges, load/topological frames, order entries, and closure membership are all +stable `int`/`i32` indices. Code reserves a graph slot before taking an element +pointer and never carries an element pointer across a graph reserve. Capacity, +addresses, request order, product order, output names, and workdir location +therefore cannot enter action identity, sorting, diagnostics, storage locators, +or artifact bytes. Dependency lists retain byte-sorted insertion and duplicate +elimination. The iterative loader retains mark-before-child and post-child +command-import validation; the iterative tri-color DFS retains deterministic +postorder and the complete live path for cycle diagnostics. + +No fixed package, product, context, action, support-map, traversal, order, or +closure cardinality remains in either driver. The unrelated `SEP_MAXLFLAGS == +32` limit is retained solely for the existing `-L`/`-l` command-line interface; +it neither indexes nor bounds package actions. Compiler and linker tools already +allocate their import/input tables from `argc`; their genuine remaining process +boundary is the host's executable-argument limit. + +The native package observers generate rather than commit large fixture trees. +The extended `long_shared_link_closure_is_complete` builds and runs a chain of +300 ordinary directory packages under independent cold Cstage and WWstage work +roots. Its command root directly imports all 300 packages and repeats one import, +proving an action beyond index 256 compiles, the root receives exactly 300 +sorted/deduplicated direct `.wwi` inputs, every ordinary action receives only its +one direct export, every `.unit.ww` contains only its two byte-sorted owner +sources, and the linker receives the root plus all 300 archives exactly once and +no `.wwi`. A second command root imports only `p000`; it reuses all 300 ordinary +actions and its exact linker line still contains the root followed by the full +`p000` through `p299` transitive archive chain and runtime archive, proving that +closure construction—not the wide root's direct imports—crosses the old boundary. +The observer compares every unit, export, assembly, object, archive, and binary +across stages. A second equivalent persistent request invokes no compiler or +assembler. Changing `p257`'s export recompiles exactly `p257`, direct importer +`p256`, and the wide root, and stops before `p255` after `p256` regenerates a +byte-identical export. Closing the chain at `p299 -> p000` produces the complete +stage-identical 300-node cycle diagnostic with empty compiler, assembler, and +linker traces. + +`dynamic_package_universe_crosses_former_boundary` first selects 309 package-test +products through 257 directory spellings in one direct request per stage. +Fifty-two real directories each produce ordinary production, internal +production-plus-test, external `_test`, and two generated-main actions; the +shared support closure brings that command-global universe to 270 actions. Two +hundred five explicit symlink spellings of `p000` add distinct valid products +and resolution contexts while reusing its one canonical internal action. The +observer reverses product and variant order between stages, proves one compile +per action, checks all 309 binaries exist, runs boundary products, validates +variant-owned units, and compares every action artifact plus representative +binaries byte-for-byte. + +The same observer then exercises the public `ww test /...` coordinator +path. The generated tree has 257 real directories: the 52 test-bearing +directories above plus 205 production-only directories. The coordinator forms +and passes 309 products and 258 contexts (including support) in one driver +request. Reusing the direct request's persistent workdir preserves all 270 +existing actions and adds exactly 410 production/generated-main actions, for 680 +distinct actions. Both stages run every product, produce byte-identical ordered +coordinator output and all 680 action artifacts, and perform no compilation on a +second equivalent public request. It checks all 104 test-bearing result labels +and all 205 no-test result labels, while a builder-boundary observer records +exactly one invocation containing all 309 descriptors for each cold and warm +public request. Existing focused observers continue to prove +dependency-first/root-first canonical reuse, root-only/combined artifact +identity, exact reordered-product trace bytes, and persistent request-directory +stability. + +This representation follows the semantic separation and scalable action +construction in the pinned official Go 1.26.5 source, identified by +[`VERSION`, lines 1–2](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/VERSION#L1-L2), +at commit `c19862e5f8415b4f24b189d065ed739517c548ba`: + +- Go's loader states that repeated package lookup returns the same pointer + ([`pkg.go`, lines 633–636](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L633-L636)), + resolves canonical path and directory before package-data lookup + ([lines 863–911](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L863-L911)), + and reuses the package cached under the resolved `ImportPath` + ([lines 757–768](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L757-L768)). +- A Go builder has one command-global action cache, while each action's + dependencies are a dynamically accumulated slice + ([`action.go`, lines 38–45](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L38-L45), + [lines 84–89](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L84-L89)). + The cache key is operation plus canonical package pointer and returns the + existing action + ([lines 202–206](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L202-L206), + [lines 437–447](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L437-L447)). +- Go constructs an archive compile action and dynamically appends actions only + for the package's direct imports + ([lines 628–659](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L628-L659)). + It separately interns a link action rooted in that cached compile action + ([lines 919–958](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L919-L958)) + and dynamically expands the complete transitive link closure + ([lines 1034–1068](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L1034-L1068)). + Requested package actions are likewise accumulated with `append` + ([`build.go`, lines 519–534](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L519-L534), + [lines 551–558](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L551-L558)). +- During execution, Go builds one package from its own source list + ([`exec.go`, lines 721–790](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L721-L790), + [lines 928–935](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L928-L935)), + maps its direct action dependencies into compiler inputs + ([lines 864–884](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L864-L884)), + and links the root archive with mappings for the complete link-action closure + ([lines 1592–1647](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1592-L1647)). +- `go/build` keeps directory, import identity, declared name, and ordinary, + internal-test, and external-test file lists separate + ([`build.go`, lines 436–493](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L436-L493)); + `ImportDir` explicitly processes the named directory + ([lines 521–525](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L521-L525)), + reads precisely that directory + ([lines 859–900](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L859-L900)), + and assigns accepted files to the separate package-owned lists + ([lines 948–1039](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L948-L1039)). +- Go's test loader explicitly returns generated main, internal production-plus- + test, and external-test packages, reusing production when valid + ([`test.go`, lines 85–102](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L85-L102)); + constructs the internal, external, and generated-main variants separately + ([lines 175–293](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L175-L293)); + dynamically appends, sorts, and deduplicates generated-main imports + ([lines 315–376](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L315-L376)); + and uses copy-on-write test variants while preserving unaffected package + objects + ([lines 421–474](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L421-L474)). + +WW adopts those package/action distinctions and scalable dependency +accumulation, but not Go's build IDs, module system, importcfg, cache/CAS, +preloader, parallel action scheduler, or network behavior. Normal local WW +builds and tests remain offline, manifest-free, registry-free, database-free, +CAS-free, and network-free. + ## 12. Candidate architectures and hard-gate decision Five candidates were developed as coherent systems, not as feature bins. diff --git a/test/package/package_test.ww b/test/package/package_test.ww index deb47cdc..b4238a7d 100644 --- a/test/package/package_test.ww +++ b/test/package/package_test.ww @@ -73,6 +73,16 @@ fn writefile(path: str, content: str) void = { assert(os.close(fd) == 0); }; +fn rewritefile(path: str, content: str) void = { + let fd: i32 = os.open(path, os.flag.WRONLY | os.flag.TRUNC, 384i32); + assert(fd >= 0); + match (os.writeall(fd, content.ptr, content.len: u64)) { + case let n: i64 => assert(n == content.len: i64); + case let e: os.oserror => abort("write failed"); + }; + assert(os.close(fd) == 0); +}; + fn writeexecutable(path: str, content: str) void = { let fd: i32 = os.open(path, os.flag.WRONLY | os.flag.CREATE | os.flag.EXCL, 448i32); @@ -202,6 +212,16 @@ fn occurrences(haystack: str, needle: str) i32 = { return count; }; +fn boundarypkgname(n: i32) str = { + assert(n >= 0 && n < 1000); + let out: []u8 = alloc([], 4u64)!; + append(out, 'p'); + append(out, ('0': i32 + (n / 100)): u8); + append(out, ('0': i32 + ((n / 10) % 10)): u8); + append(out, ('0': i32 + (n % 10)): u8); + return strings.frombytes(out); +}; + fn linecontaining(text: str, needle: str) str = { let start: i32 = 0; let i: i32 = 0; @@ -353,6 +373,8 @@ fn packagestoragekey(path: str, canon: str, variant: i32, role: i32) str = { }; fn requestworkkey(dir: str) str = { + let escaped: str = strings.concat("d_", workescape(dir)); + if (escaped.len <= 255) { return escaped; }; let state: sha256.state = sha256.sha256(); let h: *hash.hash = (&state): *hash.hash; hash.write(h, strings.toutf8("ww-request-workdir-v1:")); @@ -2381,104 +2403,820 @@ fn mkdirall(path: str) void = { let d3: str = strings.concat(d2, "/cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"); let target: str = strings.concat(d3, "/target"); + let chainroot: str = strings.concat(d3, "/chain"); assert(os.mkdir(d1, 448i32) == 0); assert(os.mkdir(d2, 448i32) == 0); assert(os.mkdir(d3, 448i32) == 0); assert(os.mkdir(target, 448i32) == 0); - let names: []str = ["p00", "p01", "p02", "p03", "p04", "p05", - "p06", "p07", "p08", "p09", "p10", "p11", "p12", "p13"]; + assert(os.mkdir(chainroot, 448i32) == 0); + let packagecount: i32 = 300; let i: i32 = 0; - for (i < names.len) { - let dir: str = strings.concat(root, "/", names[i]); + for (i < packagecount) { + let name: str = boundarypkgname(i); + let dir: str = strings.concat(root, "/", name); assert(os.mkdir(dir, 448i32) == 0); - let source: str = ""; - if (i + 1 < names.len) { - source = strings.concat("package ", names[i], ";\nimport ", - names[i + 1], ";\nexport fn value() i32 = { return ", - names[i + 1], ".value(); };\n"); + let a: str = ""; + if (i + 1 < packagecount) { + let next: str = boundarypkgname(i + 1); + a = strings.concat("package ", name, ";\nimport ", next, + ";\n// OWNER_A_", name, + "\nexport fn value() i32 = { return ", next, + ".value(); };\n"); } else { - source = strings.concat("package ", names[i], - ";\nexport fn value() i32 = { return 42; };\n"); + a = strings.concat("package ", name, ";\n// OWNER_A_", name, + "\nexport fn value() i32 = { return 42; };\n"); }; - writefile(strings.concat(dir, "/", names[i], ".ww"), source); + let z: str = strings.concat("package ", name, ";\n// OWNER_Z_", + name, "\nfn private_value() i32 = { return value(); };\n"); + writefile(strings.concat(dir, "/a.ww"), a); + writefile(strings.concat(dir, "/z.ww"), z); i += 1; }; - writefile(strings.concat(target, "/target.ww"), strings.concat( - "package target;\nimport p00;\n", - "fn value() i32 = { return p00.value(); };\n")); - writefile(strings.concat(target, "/target_test.ww"), strings.concat( - "package target;\n@test fn complete_long_closure() void = {", - " assert(value() == 42); };\n")); - let trace: str = strings.concat(root, "/long-link.trace"); - let wrapper: str = strings.concat(root, "/long-w6l.sh"); - writefile(trace, ""); - writeexecutable(wrapper, strings.concat( - "#!/bin/sh\nfor arg in \"$@\"; do printf '%s\\n' \"$arg\"; done", - " >> \"$WW_LONG_LINK_TRACE\"\nexec \"$WW_LONG_W6L\" \"$@\"\n")); + let roota: str = "package main;\n// ROOT_A\n"; + i = 0; + for (i < packagecount) { + roota = strings.concat(roota, "import ", boundarypkgname(i), ";\n"); + i += 1; + }; + let rootz: str = strings.concat("package main;\nimport p150;\n", + "// ROOT_Z\nfn main() i32 = { return p000.value() - 42; };\n"); + writefile(strings.concat(target, "/a.ww"), roota); + writefile(strings.concat(target, "/z.ww"), rootz); + let chainsource: str = strings.concat("package main;\nimport p000;\n", + "// CHAIN_ROOT\nfn main() i32 = { return p000.value() - 42; };\n"); + writefile(strings.concat(chainroot, "/main.ww"), chainsource); + let rootaction: str = "target"; + let chainaction: str = "chain"; + + let compilerwrapper: str = strings.concat(root, "/boundary-w6c.sh"); + let assemblerwrapper: str = strings.concat(root, "/boundary-w6a.sh"); + let linkerwrapper: str = strings.concat(root, "/boundary-w6l.sh"); + writeexecutable(compilerwrapper, strings.concat( + "#!/bin/sh\nprintf 'BEGIN' >> \"$WW_BOUNDARY_COMPILER_TRACE\"\n", + "for arg in \"$@\"; do printf '<%s>' \"$arg\" >> ", + "\"$WW_BOUNDARY_COMPILER_TRACE\"; done\n", + "printf '\\n' >> \"$WW_BOUNDARY_COMPILER_TRACE\"\n", + "exec \"$WW_BOUNDARY_W6C\" \"$@\"\n")); + writeexecutable(assemblerwrapper, strings.concat( + "#!/bin/sh\nprintf 'BEGIN' >> \"$WW_BOUNDARY_ASSEMBLER_TRACE\"\n", + "for arg in \"$@\"; do printf '<%s>' \"$arg\" >> ", + "\"$WW_BOUNDARY_ASSEMBLER_TRACE\"; done\n", + "printf '\\n' >> \"$WW_BOUNDARY_ASSEMBLER_TRACE\"\n", + "exec \"$WW_BOUNDARY_W6A\" \"$@\"\n")); + writeexecutable(linkerwrapper, strings.concat( + "#!/bin/sh\nprintf 'BEGIN' >> \"$WW_BOUNDARY_LINKER_TRACE\"\n", + "for arg in \"$@\"; do printf '<%s>' \"$arg\" >> ", + "\"$WW_BOUNDARY_LINKER_TRACE\"; done\n", + "printf '\\n' >> \"$WW_BOUNDARY_LINKER_TRACE\"\n", + "exec \"$WW_BOUNDARY_W6L\" \"$@\"\n")); + let stages: []str = ["ww", "ww_ww"]; + let compilers: []str = ["w6c", "w6c_ww"]; + let assemblers: []str = ["w6a", "w6a_ww"]; + let linkers: []str = ["w6l", "w6l_ww"]; + let works: []str = [strings.concat(root, "/boundary-c-work"), + strings.concat(root, "/boundary-ww-work")]; + let bins: []str = [strings.concat(root, "/boundary-c-bin"), + strings.concat(root, "/boundary-ww-bin")]; + let chainbins: []str = [strings.concat(root, "/boundary-c-chain-bin"), + strings.concat(root, "/boundary-ww-chain-bin")]; + let compilertraces: []str = [strings.concat(root, "/boundary-c-compiler"), + strings.concat(root, "/boundary-ww-compiler")]; + let assemblertraces: []str = [strings.concat(root, "/boundary-c-assembler"), + strings.concat(root, "/boundary-ww-assembler")]; + let linkertraces: []str = [strings.concat(root, "/boundary-c-linker"), + strings.concat(root, "/boundary-ww-linker")]; let baseenv: []str = os.getenvs(); - let env: []str = alloc([], (baseenv.len + 3): u64)!; - let ei: i32 = 0; - for (ei < baseenv.len) { - if (!strings.hasprefix(baseenv[ei], "WW_W6L=") - && !strings.hasprefix(baseenv[ei], "WW_LONG_LINK_TRACE=") - && !strings.hasprefix(baseenv[ei], "WW_LONG_W6L=")) { - append(env, baseenv[ei]); - }; - ei += 1; - }; - append(env, strings.concat("WW_W6L=", wrapper)); - append(env, strings.concat("WW_LONG_LINK_TRACE=", trace)); - append(env, strings.concat("WW_LONG_W6L=", driver("w6l"))); - let bin: str = strings.concat(target, "/target.test"); - let workroot: str = strings.concat(bin, ".sepwork"); - let work: str = strings.concat(workroot, "/"); - let cargv: []str = [driver("ww"), "test", "-c", "-I", d3, - "-I", root, target]; + let environments: [][]str = alloc([], stages.len: u64)!; let out: commandout; - runcommandenv(root, "long-link-c", cargv, env, - (120i64 * (time.second: i64)): time.duration, &out); - expectexit(&out, 0); - let cstdout: str = strings.dup(out.stdout); - let cstderr: str = strings.dup(out.stderr); - let cbin: str = readfile(bin); - let rootunit: str = readfile(strings.concat(work, - "target-internal-test.unit.ww")); - assert(has(rootunit, "//ww:module-reset target\npackage target;")); - assert(!has(rootunit, "//ww:module ")); - let linkargs: str = readfile(trace); - assert(linkargs.len > 8192); - assert(!has(linkargs, ".wwi\n")); + let si: i32 = 0; + for (si < stages.len) { + assert(os.mkdir(works[si], 448i32) == 0); + writefile(compilertraces[si], ""); + writefile(assemblertraces[si], ""); + writefile(linkertraces[si], ""); + let env: []str = alloc([], (baseenv.len + 11): u64)!; + let ei: i32 = 0; + for (ei < baseenv.len) { + if (!strings.hasprefix(baseenv[ei], "WW_W6C=") + && !strings.hasprefix(baseenv[ei], "WW_W6A=") + && !strings.hasprefix(baseenv[ei], "WW_W6L=") + && !strings.hasprefix(baseenv[ei], + "WW_BOUNDARY_COMPILER_TRACE=") + && !strings.hasprefix(baseenv[ei], + "WW_BOUNDARY_ASSEMBLER_TRACE=") + && !strings.hasprefix(baseenv[ei], + "WW_BOUNDARY_LINKER_TRACE=") + && !strings.hasprefix(baseenv[ei], "WW_BOUNDARY_W6C=") + && !strings.hasprefix(baseenv[ei], "WW_BOUNDARY_W6A=") + && !strings.hasprefix(baseenv[ei], "WW_BOUNDARY_W6L=") + && !strings.hasprefix(baseenv[ei], "WW_LIB=") + && !strings.hasprefix(baseenv[ei], "WW_SRCLIB=")) { + append(env, baseenv[ei]); + }; + ei += 1; + }; + append(env, strings.concat("WW_W6C=", compilerwrapper)); + append(env, strings.concat("WW_W6A=", assemblerwrapper)); + append(env, strings.concat("WW_W6L=", linkerwrapper)); + append(env, strings.concat("WW_BOUNDARY_COMPILER_TRACE=", + compilertraces[si])); + append(env, strings.concat("WW_BOUNDARY_ASSEMBLER_TRACE=", + assemblertraces[si])); + append(env, strings.concat("WW_BOUNDARY_LINKER_TRACE=", + linkertraces[si])); + append(env, strings.concat("WW_BOUNDARY_W6C=", + driver(compilers[si]))); + append(env, strings.concat("WW_BOUNDARY_W6A=", + driver(assemblers[si]))); + append(env, strings.concat("WW_BOUNDARY_W6L=", + driver(linkers[si]))); + append(env, strings.concat("WW_LIB=", repo(), "/out/bin/../lib")); + append(env, strings.concat("WW_SRCLIB=", repo(), "/lib")); + append(environments, env); + let av: []str = [driver(stages[si]), "build", "-w", works[si], + "-I", d3, "-I", root, "-o", bins[si], target]; + runcommandenv(root, strings.concat("boundary-cold-", stages[si]), av, + env, (600i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stdout.len == 0 && out.stderr.len == 0); + let runav: []str = [bins[si]]; + runcommand(root, strings.concat("boundary-run-", stages[si]), runav, + (60i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stdout.len == 0 && out.stderr.len == 0); + + let ctrace: str = readfile(compilertraces[si]); + let atrace: str = readfile(assemblertraces[si]); + let ltrace: str = readfile(linkertraces[si]); + assert(occurrences(ctrace, "\n") == packagecount + 1); + assert(occurrences(atrace, "\n") == packagecount + 1); + assert(occurrences(ltrace, "\n") == 1); + let rootline: str = linecontaining(ctrace, + strings.concat("/", rootaction, ".unit.new")); + assert(occurrences(rootline, "<--import>") == packagecount); + let expectedroot: str = "BEGIN<--entry><-c>"; + i = 0; + for (i < packagecount) { + let name: str = boundarypkgname(i); + expectedroot = strings.concat(expectedroot, "<--import><", name, + "><", works[si], "/", name, ".wwi>"); + assert(occurrences(rootline, strings.concat("<", name, "><", + works[si], "/", name, ".wwi>")) == 1); + i += 1; + }; + expectedroot = strings.concat(expectedroot, "<-I><", works[si], "/", + rootaction, ".wwi.new><-o><", works[si], "/", rootaction, + ".s.new><", works[si], "/", rootaction, ".unit.new>"); + assert(same(rootline, expectedroot)); + i = 0; + for (i < packagecount) { + let name: str = boundarypkgname(i); + let expectedcompile: str = "BEGIN<-c>"; + if (i + 1 < packagecount) { + let next: str = boundarypkgname(i + 1); + expectedcompile = strings.concat(expectedcompile, + "<--import><", next, "><", works[si], "/", next, + ".wwi>"); + }; + expectedcompile = strings.concat(expectedcompile, "<-I><", + works[si], "/", name, ".wwi.new><-o><", works[si], "/", + name, ".s.new><", works[si], "/", name, ".unit.new>"); + assert(same(linecontaining(ctrace, + strings.concat("/", name, ".unit.new")), expectedcompile)); + let expectedassemble: str = strings.concat("BEGIN<-o><", works[si], + "/", name, ".o.new><", works[si], "/", name, ".s.new>"); + assert(same(linecontaining(atrace, + strings.concat("/", name, ".o.new")), expectedassemble)); + assert(occurrences(ltrace, strings.concat("<", works[si], "/", + name, ".a>")) == 1); + assert(os.exists(strings.concat(works[si], "/", name, ".wwi"))); + assert(os.exists(strings.concat(works[si], "/", name, ".s"))); + assert(os.exists(strings.concat(works[si], "/", name, ".o"))); + assert(os.exists(strings.concat(works[si], "/", name, ".a"))); + let a: str = ""; + if (i + 1 < packagecount) { + let next: str = boundarypkgname(i + 1); + a = strings.concat("package ", name, ";\nimport ", next, + ";\n// OWNER_A_", name, + "\nexport fn value() i32 = { return ", next, + ".value(); };\n"); + } else { + a = strings.concat("package ", name, ";\n// OWNER_A_", name, + "\nexport fn value() i32 = { return 42; };\n"); + }; + let z: str = strings.concat("package ", name, ";\n// OWNER_Z_", + name, "\nfn private_value() i32 = { return value(); };\n"); + assert(same(readfile(strings.concat(works[si], "/", name, + ".unit.ww")), strings.concat("//ww:module-reset ", name, + "\n", a, "\n//ww:module-reset ", name, "\n", z, "\n"))); + assert(strings.hasprefix(readfile(strings.concat(works[si], "/", + name, ".wwi")), strings.concat("//ww:module ", name, "\n"))); + i += 1; + }; + assert(occurrences(ltrace, strings.concat("<", works[si], "/", + rootaction, ".a>")) == 1); + assert(!has(ltrace, ".wwi>")); + let expectedlink: str = strings.concat("BEGIN<-o><", bins[si], "><", + works[si], "/", rootaction, ".a>"); + i = 0; + for (i < packagecount) { + expectedlink = strings.concat(expectedlink, "<", works[si], "/", + boundarypkgname(i), ".a>"); + i += 1; + }; + expectedlink = strings.concat(expectedlink, "<", repo(), + "/out/bin/../lib/libwwrt.a>\n"); + assert(same(ltrace, expectedlink)); + assert(same(readfile(strings.concat(works[si], "/", rootaction, + ".unit.ww")), + strings.concat("//ww:module-reset target\n", roota, + "\n//ww:module-reset target\n", rootz, "\n"))); + let rootassemble: str = strings.concat("BEGIN<-o><", works[si], "/", + rootaction, ".o.new><", works[si], "/", rootaction, ".s.new>"); + assert(same(linecontaining(atrace, + strings.concat("/", rootaction, ".o.new")), rootassemble)); + + // Repeating the same semantic request in the persistent work root may + // relink the requested binary, but must not compile or assemble any action. + let cbefore: i32 = ctrace.len; + let abefore: i32 = atrace.len; + runcommandenv(root, strings.concat("boundary-warm-", stages[si]), av, + env, (600i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stdout.len == 0 && out.stderr.len == 0); + assert(readfile(compilertraces[si]).len == cbefore); + assert(readfile(assemblertraces[si]).len == abefore); + + // A second root importing only p000 proves that link closure expansion, + // not the wide root's direct import list, reaches every archive beyond 256. + let chaincbefore: i32 = readfile(compilertraces[si]).len; + let chainabefore: i32 = readfile(assemblertraces[si]).len; + let chainlbefore: i32 = readfile(linkertraces[si]).len; + let chainav: []str = [driver(stages[si]), "build", "-w", works[si], + "-I", d3, "-I", root, "-o", chainbins[si], chainroot]; + runcommandenv(root, strings.concat("boundary-chain-", stages[si]), chainav, + env, (600i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stdout.len == 0 && out.stderr.len == 0); + let chainctrace: str = readfile(compilertraces[si]); + let chainatrace: str = readfile(assemblertraces[si]); + let chainltrace: str = readfile(linkertraces[si]); + let cdelta: str = strings.sub(chainctrace, chaincbefore, chainctrace.len); + let adelta: str = strings.sub(chainatrace, chainabefore, chainatrace.len); + let ldelta: str = strings.sub(chainltrace, chainlbefore, chainltrace.len); + assert(occurrences(cdelta, "\n") == 1); + assert(occurrences(adelta, "\n") == 1); + assert(occurrences(ldelta, "\n") == 1); + assert(same(cdelta, strings.concat("BEGIN<--entry><-c><--import><", + works[si], "/p000.wwi><-I><", works[si], "/", chainaction, + ".wwi.new><-o><", works[si], "/", chainaction, ".s.new><", + works[si], "/", chainaction, ".unit.new>\n"))); + assert(same(adelta, strings.concat("BEGIN<-o><", works[si], "/", + chainaction, ".o.new><", works[si], "/", chainaction, ".s.new>\n"))); + let expectedchainlink: str = strings.concat("BEGIN<-o><", chainbins[si], + "><", works[si], "/", chainaction, ".a>"); + i = 0; + for (i < packagecount) { + expectedchainlink = strings.concat(expectedchainlink, "<", works[si], + "/", boundarypkgname(i), ".a>"); + i += 1; + }; + expectedchainlink = strings.concat(expectedchainlink, "<", repo(), + "/out/bin/../lib/libwwrt.a>\n"); + assert(same(ldelta, expectedchainlink)); + assert(!has(ldelta, ".wwi>")); + assert(same(readfile(strings.concat(works[si], "/", chainaction, + ".unit.ww")), strings.concat("//ww:module-reset chain\n", + chainsource, "\n"))); + let chainrun: []str = [chainbins[si]]; + runcommand(root, strings.concat("boundary-chain-run-", stages[si]), + chainrun, (60i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + si += 1; + }; + + let suffixes: []str = [".unit.ww", ".wwi", ".s", ".o", ".a"]; i = 0; - for (i < names.len) { - assert(os.exists(strings.concat(work, names[i], ".wwi"))); - assert(os.exists(strings.concat(work, names[i], ".a"))); - assert(has(linkargs, strings.concat(work, names[i], ".a\n"))); + for (i < packagecount) { + let name: str = boundarypkgname(i); + let xi: i32 = 0; + for (xi < suffixes.len) { + assert(same(readfile(strings.concat(works[0], "/", name, + suffixes[xi])), readfile(strings.concat(works[1], "/", name, + suffixes[xi])))); + xi += 1; + }; i += 1; }; - assert(has(linkargs, strings.concat(work, "test.a\n"))); - let runav: []str = [bin]; - runcommand(root, "long-link-run-c", runav, - (60i64 * (time.second: i64)): time.duration, &out); - expectexit(&out, 0); - assert(has(out.stdout, "complete_long_closure ... ok\n")); - clean(workroot); clean(bin); - let wargv: []str = [driver("ww_ww"), "test", "-c", "-I", d3, - "-I", root, target]; - runcommand(root, "long-link-ww", wargv, - (120i64 * (time.second: i64)): time.duration, &out); - expectexit(&out, 0); - assert(same(cstdout, out.stdout)); - assert(same(cstderr, out.stderr)); - assert(same(cbin, readfile(bin))); + let xi: i32 = 0; + for (xi < suffixes.len) { + assert(same(readfile(strings.concat(works[0], "/", rootaction, + suffixes[xi])), readfile(strings.concat(works[1], "/", rootaction, + suffixes[xi])))); + xi += 1; + }; + assert(same(readfile(bins[0]), readfile(bins[1]))); + xi = 0; + for (xi < suffixes.len) { + assert(same(readfile(strings.concat(works[0], "/", chainaction, + suffixes[xi])), readfile(strings.concat(works[1], "/", chainaction, + suffixes[xi])))); + xi += 1; + }; + assert(same(readfile(chainbins[0]), readfile(chainbins[1]))); + + // p257 is inserted beyond the former 256-action boundary. A changed + // export recompiles that package, its direct importer p256, and the root + // (which directly imports every package), then stops at p255 because the + // regenerated p256 export is byte-identical. + let changed: str = strings.concat(root, "/p257/a.ww"); + rewritefile(changed, strings.concat( + "package p257;\nimport p258;\n// OWNER_A_p257_CHANGED\n", + "export fn value() i32 = { return p258.value(); };\n", + "export fn changed() i32 = { return 257; };\n")); + si = 0; + for (si < stages.len) { + let before: str = readfile(compilertraces[si]); + let av: []str = [driver(stages[si]), "build", "-w", works[si], + "-I", d3, "-I", root, "-o", bins[si], target]; + runcommandenv(root, strings.concat("boundary-change-", stages[si]), av, + environments[si], + (600i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stdout.len == 0 && out.stderr.len == 0); + let after: str = readfile(compilertraces[si]); + let delta: str = strings.sub(after, before.len, after.len); + assert(occurrences(delta, "\n") == 3); + assert(occurrences(delta, "/p257.unit.new") == 1); + assert(occurrences(delta, "/p256.unit.new") == 1); + assert(occurrences(delta, strings.concat("/", rootaction, + ".unit.new")) == 1); + assert(!has(delta, "/p255.unit.new")); + assert(!has(delta, "/p258.unit.new")); + let runav: []str = [bins[si]]; + runcommand(root, strings.concat("boundary-change-run-", stages[si]), + runav, (60i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + si += 1; + }; + assert(same(readfile(strings.concat(works[0], "/p257.wwi")), + readfile(strings.concat(works[1], "/p257.wwi")))); + assert(same(readfile(bins[0]), readfile(bins[1]))); + + // Close the 300-node chain into a cycle. Both graph implementations must + // emit the complete deterministic path before compiler, assembler, or linker. + rewritefile(strings.concat(root, "/p299/a.ww"), strings.concat( + "package p299;\nimport p000;\n// OWNER_A_p299_CYCLE\n", + "export fn value() i32 = { return p000.value(); };\n")); + let wantcycle: str = "ww: dependency cycle: "; i = 0; - for (i < names.len) { - assert(os.exists(strings.concat(work, names[i], ".a"))); + for (i < packagecount) { + if (i != 0) { wantcycle = strings.concat(wantcycle, " -> "); }; + wantcycle = strings.concat(wantcycle, boundarypkgname(i)); i += 1; }; - runcommand(root, "long-link-run-ww", runav, - (60i64 * (time.second: i64)): time.duration, &out); - expectexit(&out, 0); - assert(has(out.stdout, "complete_long_closure ... ok\n")); + wantcycle = strings.concat(wantcycle, " -> p000\n"); + let referencecycle: str = ""; + si = 0; + for (si < stages.len) { + rewritefile(compilertraces[si], ""); + rewritefile(assemblertraces[si], ""); + rewritefile(linkertraces[si], ""); + let cyclework: str = strings.concat(root, "/cycle-work-", stages[si]); + assert(os.mkdir(cyclework, 448i32) == 0); + let cyclebin: str = strings.concat(root, "/cycle-bin-", stages[si]); + let av: []str = [driver(stages[si]), "build", "-w", cyclework, + "-I", d3, "-I", root, "-o", cyclebin, target]; + runcommandenv(root, strings.concat("boundary-cycle-", stages[si]), av, + environments[si], + (600i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 1); + assert(out.stdout.len == 0); + assert(same(out.stderr, wantcycle)); + assert(readfile(compilertraces[si]).len == 0); + assert(readfile(assemblertraces[si]).len == 0); + assert(readfile(linkertraces[si]).len == 0); + if (si == 0) { referencecycle = strings.dup(out.stderr); } + else { assert(same(referencecycle, out.stderr)); }; + si += 1; + }; + clean(root); +}; + +@test fn dynamic_package_universe_crosses_former_boundary() void = { + let root: str = fresh(); + let suite: str = strings.concat(root, "/suite"); + assert(os.mkdir(suite, 448i32) == 0); + let directorycount: i32 = 52; + let aliascount: i32 = 205; + let totaldirectorycount: i32 = directorycount + aliascount; + let productcount: i32 = directorycount * 2 + aliascount; + // Each selected directory contributes production, internal, external, and + // two generated-main actions. The shared test support action and its nine + // ordinary dependencies are command-global actions too. + let directactioncount: i32 = directorycount * 5 + 10; + let coordinatoractioncount: i32 = directactioncount + aliascount * 2; + let i: i32 = 0; + for (i < directorycount) { + let name: str = boundarypkgname(i); + let dir: str = strings.concat(suite, "/", name); + assert(os.mkdir(dir, 448i32) == 0); + writefile(strings.concat(dir, "/a.ww"), strings.concat( + "package ", name, ";\n// DYNAMIC_PRODUCTION_", name, + "\nexport fn value() i32 = { return 42; };\n")); + writefile(strings.concat(dir, "/same_test.ww"), strings.concat( + "package ", name, ";\n// DYNAMIC_INTERNAL_", name, + "\n@test fn same_value() void = { assert(value() == 42); };\n")); + writefile(strings.concat(dir, "/external_test.ww"), strings.concat( + "package ", name, "_test;\nimport ", name, + ";\n// DYNAMIC_EXTERNAL_", name, + "\n@test fn external_value() void = { assert(", name, + ".value() == 42); };\n")); + i += 1; + }; + // Production-only directories make the recursive coordinator request own + // 257 resolution contexts and 309 products. Each contributes a production + // action plus its generated production test-main action. + for (i < totaldirectorycount) { + let name: str = boundarypkgname(i); + let dir: str = strings.concat(suite, "/", name); + assert(os.mkdir(dir, 448i32) == 0); + writefile(strings.concat(dir, "/a.ww"), strings.concat( + "package ", name, ";\n// DYNAMIC_PRODUCTION_ONLY_", name, + "\nexport fn value() i32 = { return 42; };\n")); + i += 1; + }; + i = 0; + for (i < aliascount) { + let alias: str = strings.concat(suite, "/alias-", boundarypkgname(i)); + assert(os.symlink(strings.concat(suite, "/p000"), alias) == 0); + i += 1; + }; + + let compilerwrapper: str = strings.concat(root, "/universe-w6c.sh"); + let linkerwrapper: str = strings.concat(root, "/universe-w6l.sh"); + writeexecutable(compilerwrapper, strings.concat( + "#!/bin/sh\nprintf '%s\\n' \"$*\" >> ", + "\"$WW_UNIVERSE_COMPILER_TRACE\"\n", + "exec \"$WW_UNIVERSE_W6C\" \"$@\"\n")); + writeexecutable(linkerwrapper, strings.concat( + "#!/bin/sh\nprintf '%s\\n' \"$*\" >> ", + "\"$WW_UNIVERSE_LINKER_TRACE\"\n", + "exec \"$WW_UNIVERSE_W6L\" \"$@\"\n")); + let coordinatorproxy: str = strings.concat(root, + "/universe-coordinator.sh"); + let driverwrapper: str = strings.concat(root, "/universe-driver.sh"); + writeexecutable(coordinatorproxy, strings.concat( + "#!/bin/sh\n", + "if [ \"$1\" != package ] || [ \"$2\" != --ww-driver ] || ", + "[ \"$#\" -lt 3 ]; then exit 97; fi\n", + "shift 3\nexec \"$WW_UNIVERSE_WWTEST\" package --ww-driver ", + "\"$WW_UNIVERSE_DRIVER_WRAPPER\" \"$@\"\n")); + writeexecutable(driverwrapper, strings.concat( + "#!/bin/sh\ncount=0\n", + "for arg in \"$@\"; do\n", + " if [ \"$arg\" = --ww-package-test ]; then count=$((count + 1)); fi\n", + "done\nprintf '%s\\n' \"$count\" >> ", + "\"$WW_UNIVERSE_BUILDER_TRACE\"\n", + "exec \"$WW_UNIVERSE_DRIVER\" \"$@\"\n")); + let stages: []str = ["ww", "ww_ww"]; + let compilers: []str = ["w6c", "w6c_ww"]; + let assemblers: []str = ["w6a", "w6a_ww"]; + let linkers: []str = ["w6l", "w6l_ww"]; + let workroots: []str = [strings.concat(root, "/universe-c-workroot"), + strings.concat(root, "/universe-ww-workroot")]; + let works: []str = [strings.concat(workroots[0], "/", requestworkkey(suite)), + strings.concat(workroots[1], "/", requestworkkey(suite))]; + let compilertraces: []str = [strings.concat(root, "/universe-c-compiler"), + strings.concat(root, "/universe-ww-compiler")]; + let linkertraces: []str = [strings.concat(root, "/universe-c-linker"), + strings.concat(root, "/universe-ww-linker")]; + let buildertraces: []str = [strings.concat(root, "/universe-c-builder"), + strings.concat(root, "/universe-ww-builder")]; + let baseenv: []str = os.getenvs(); + let coordinatoroutputs: []str = ["", ""]; + let out: commandout; + let si: i32 = 0; + for (si < stages.len) { + assert(os.mkdir(workroots[si], 448i32) == 0); + assert(os.mkdir(works[si], 448i32) == 0); + writefile(compilertraces[si], ""); + writefile(linkertraces[si], ""); + writefile(buildertraces[si], ""); + let env: []str = alloc([], (baseenv.len + 14): u64)!; + let ei: i32 = 0; + for (ei < baseenv.len) { + if (!strings.hasprefix(baseenv[ei], "WW_W6C=") + && !strings.hasprefix(baseenv[ei], "WW_W6A=") + && !strings.hasprefix(baseenv[ei], "WW_W6L=") + && !strings.hasprefix(baseenv[ei], + "WW_UNIVERSE_COMPILER_TRACE=") + && !strings.hasprefix(baseenv[ei], + "WW_UNIVERSE_LINKER_TRACE=") + && !strings.hasprefix(baseenv[ei], "WW_UNIVERSE_W6C=") + && !strings.hasprefix(baseenv[ei], "WW_UNIVERSE_W6L=") + && !strings.hasprefix(baseenv[ei], "WW_UNIVERSE_WWTEST=") + && !strings.hasprefix(baseenv[ei], + "WW_UNIVERSE_DRIVER_WRAPPER=") + && !strings.hasprefix(baseenv[ei], "WW_UNIVERSE_DRIVER=") + && !strings.hasprefix(baseenv[ei], + "WW_UNIVERSE_BUILDER_TRACE=") + && !strings.hasprefix(baseenv[ei], "WW_WWTEST=") + && !strings.hasprefix(baseenv[ei], "WW_LIB=") + && !strings.hasprefix(baseenv[ei], "WW_SRCLIB=")) { + append(env, baseenv[ei]); + }; + ei += 1; + }; + append(env, strings.concat("WW_W6C=", compilerwrapper)); + append(env, strings.concat("WW_W6A=", driver(assemblers[si]))); + append(env, strings.concat("WW_W6L=", linkerwrapper)); + append(env, strings.concat("WW_UNIVERSE_COMPILER_TRACE=", + compilertraces[si])); + append(env, strings.concat("WW_UNIVERSE_LINKER_TRACE=", + linkertraces[si])); + append(env, strings.concat("WW_UNIVERSE_W6C=", + driver(compilers[si]))); + append(env, strings.concat("WW_UNIVERSE_W6L=", driver(linkers[si]))); + append(env, strings.concat("WW_LIB=", repo(), "/out/bin/../lib")); + append(env, strings.concat("WW_SRCLIB=", repo(), "/lib")); + append(env, strings.concat("WW_WWTEST=", coordinatorproxy)); + append(env, strings.concat("WW_UNIVERSE_WWTEST=", repo(), + "/out/bin/wwtest")); + append(env, strings.concat("WW_UNIVERSE_DRIVER_WRAPPER=", + driverwrapper)); + append(env, strings.concat("WW_UNIVERSE_DRIVER=", driver(stages[si]))); + append(env, strings.concat("WW_UNIVERSE_BUILDER_TRACE=", + buildertraces[si])); + + let av: []str = alloc([], (8 + productcount * 6): u64)!; + append(av, driver(stages[si])); + append(av, "test"); append(av, "-c"); + append(av, "-w"); append(av, works[si]); + append(av, "-I"); append(av, suite); + if (si == 0) { + i = 0; + for (i < directorycount) { + let name: str = boundarypkgname(i); + let dir: str = strings.concat(suite, "/", name); + append(av, "--ww-package-test"); append(av, "same"); + append(av, name); append(av, dir); + append(av, strings.concat(root, "/c-same-", name)); + append(av, strings.concat(root, "/c-same-", name, ".status")); + append(av, "--ww-package-test"); append(av, "external"); + append(av, strings.concat(name, "_test")); append(av, dir); + append(av, strings.concat(root, "/c-external-", name)); + append(av, strings.concat(root, "/c-external-", name, + ".status")); + i += 1; + }; + i = 0; + for (i < aliascount) { + let aname: str = boundarypkgname(i); + let alias: str = strings.concat(suite, "/alias-", aname); + append(av, "--ww-package-test"); append(av, "same"); + append(av, "p000"); append(av, alias); + append(av, strings.concat(root, "/c-alias-", aname)); + append(av, strings.concat(root, "/c-alias-", aname, + ".status")); + i += 1; + }; + } else { + // Reverse both the product groups and variant order. The driver must + // derive the same canonical action universe independent of request order. + i = aliascount - 1; + for (i >= 0) { + let aname: str = boundarypkgname(i); + let alias: str = strings.concat(suite, "/alias-", aname); + append(av, "--ww-package-test"); append(av, "same"); + append(av, "p000"); append(av, alias); + append(av, strings.concat(root, "/ww-alias-", aname)); + append(av, strings.concat(root, "/ww-alias-", aname, + ".status")); + i -= 1; + }; + i = directorycount - 1; + for (i >= 0) { + let name: str = boundarypkgname(i); + let dir: str = strings.concat(suite, "/", name); + append(av, "--ww-package-test"); append(av, "external"); + append(av, strings.concat(name, "_test")); append(av, dir); + append(av, strings.concat(root, "/ww-external-", name)); + append(av, strings.concat(root, "/ww-external-", name, + ".status")); + append(av, "--ww-package-test"); append(av, "same"); + append(av, name); append(av, dir); + append(av, strings.concat(root, "/ww-same-", name)); + append(av, strings.concat(root, "/ww-same-", name, ".status")); + i -= 1; + }; + }; + append(av, strings.concat(suite, "/p000")); + assert(av.len == 8 + productcount * 6); + runcommandenv(root, strings.concat("dynamic-universe-", stages[si]), av, + env, (600i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stdout.len == 0 && out.stderr.len == 0); + let ctrace: str = readfile(compilertraces[si]); + let ltrace: str = readfile(linkertraces[si]); + assert(occurrences(ctrace, "\n") == directactioncount); + assert(occurrences(ltrace, "\n") == productcount); + assert(!has(ltrace, ".wwi")); + assert(occurrences(ctrace, "/test.unit.new") == 1); + i = 0; + for (i < directorycount) { + let name: str = boundarypkgname(i); + let internal: str = strings.concat(name, "-internal-test"); + let external: str = strings.concat(name, + "_test-external-test"); + let internalmain: str = strings.concat(internal, "-main"); + let externalmain: str = strings.concat(external, "-main"); + assert(occurrences(ctrace, strings.concat("/", name, + ".unit.new")) == 1); + assert(occurrences(ctrace, strings.concat("/", internal, + ".unit.new")) == 1); + assert(occurrences(ctrace, strings.concat("/", external, + ".unit.new")) == 1); + assert(occurrences(ctrace, strings.concat("/", internalmain, + ".unit.new")) == 1); + assert(occurrences(ctrace, strings.concat("/", externalmain, + ".unit.new")) == 1); + let actions: []str = [name, internal, external, internalmain, + externalmain]; + let ai: i32 = 0; + for (ai < actions.len) { + assert(os.exists(strings.concat(works[si], "/", actions[ai], + ".unit.ww"))); + assert(os.exists(strings.concat(works[si], "/", actions[ai], + ".wwi"))); + assert(os.exists(strings.concat(works[si], "/", actions[ai], + ".s"))); + assert(os.exists(strings.concat(works[si], "/", actions[ai], + ".o"))); + assert(os.exists(strings.concat(works[si], "/", actions[ai], + ".a"))); + ai += 1; + }; + let prefix: str = "c"; + if (si == 1) { prefix = "ww"; }; + assert(os.exists(strings.concat(root, "/", prefix, "-same-", name))); + assert(os.exists(strings.concat(root, "/", prefix, "-external-", + name))); + i += 1; + }; + i = 0; + for (i < aliascount) { + let name: str = boundarypkgname(i); + let prefix: str = "c"; + if (si == 1) { prefix = "ww"; }; + assert(os.exists(strings.concat(root, "/", prefix, "-alias-", name))); + i += 1; + }; + let prefix: str = "c"; + if (si == 1) { prefix = "ww"; }; + let samples: []str = [strings.concat(root, "/", prefix, "-same-p051"), + strings.concat(root, "/", prefix, "-external-p051"), + strings.concat(root, "/", prefix, "-alias-p204")]; + i = 0; + for (i < samples.len) { + let runav: []str = [samples[i]]; + runcommand(root, strings.concat("dynamic-run-", stages[si], "-", + boundarypkgname(i)), runav, + (60i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(has(out.stdout, "_value ... ok\n")); + i += 1; + }; + + // Exercise the public coordinator path with the same persistent package + // universe. It must construct and pass all 309 descriptors in one argv, + // retain the 270 existing actions, and add exactly the 410 actions owned by + // the 205 production-only directories. + let cbefore: i32 = readfile(compilertraces[si]).len; + let lbefore: i32 = readfile(linkertraces[si]).len; + let recursive: str = strings.concat(suite, "/..."); + let coordinatorav: []str = [driver(stages[si]), "test", "-j", "1", + "-w", workroots[si], "-I", suite, recursive]; + runcommandenv(root, strings.concat("dynamic-coordinator-", stages[si]), + coordinatorav, env, + (1200i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stderr.len == 0); + coordinatoroutputs[si] = strings.dup(out.stdout); + let cafter: str = readfile(compilertraces[si]); + let lafter: str = readfile(linkertraces[si]); + assert(occurrences(cafter, "\n") == coordinatoractioncount); + assert(occurrences(strings.sub(cafter, cbefore, cafter.len), "\n") + == aliascount * 2); + assert(occurrences(strings.sub(lafter, lbefore, lafter.len), "\n") + == productcount); + assert(!has(strings.sub(lafter, lbefore, lafter.len), ".wwi")); + assert(same(readfile(buildertraces[si]), "309\n")); + assert(occurrences(out.stdout, "ok ") == directorycount * 2); + assert(occurrences(out.stdout, "? ") == aliascount); + i = 0; + for (i < directorycount) { + let name: str = boundarypkgname(i); + assert(has(out.stdout, strings.concat("ok ", suite, "/", name, + " [", name, ", same-package]\n"))); + assert(has(out.stdout, strings.concat("ok ", suite, "/", name, + " [", name, "_test, external]\n"))); + i += 1; + }; + i = directorycount; + for (i < totaldirectorycount) { + let name: str = boundarypkgname(i); + assert(has(out.stdout, strings.concat("? ", suite, "/", name, + " [no tests]\n"))); + i += 1; + }; + + // Repeating the public request may relink products, but every canonical + // action in the >256 universe must be warm. + let warmcompilerlen: i32 = cafter.len; + runcommandenv(root, strings.concat("dynamic-coordinator-warm-", stages[si]), + coordinatorav, env, + (1200i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stderr.len == 0); + assert(same(out.stdout, coordinatoroutputs[si])); + assert(readfile(compilertraces[si]).len == warmcompilerlen); + assert(same(readfile(buildertraces[si]), "309\n309\n")); + si += 1; + }; + assert(same(coordinatoroutputs[0], coordinatoroutputs[1])); + + let suffixes: []str = [".unit.ww", ".wwi", ".s", ".o", ".a"]; + i = 0; + for (i < directorycount) { + let name: str = boundarypkgname(i); + let actions: []str = [name, strings.concat(name, "-internal-test"), + strings.concat(name, "_test-external-test"), + strings.concat(name, "-internal-test-main"), + strings.concat(name, "_test-external-test-main")]; + let ai: i32 = 0; + for (ai < actions.len) { + let xi: i32 = 0; + for (xi < suffixes.len) { + assert(same(readfile(strings.concat(works[0], "/", actions[ai], + suffixes[xi])), readfile(strings.concat(works[1], "/", + actions[ai], suffixes[xi])))); + xi += 1; + }; + ai += 1; + }; + i += 1; + }; + i = directorycount; + for (i < totaldirectorycount) { + let name: str = boundarypkgname(i); + let actions: []str = [name, strings.concat(name, "-main")]; + let ai: i32 = 0; + for (ai < actions.len) { + let xi: i32 = 0; + for (xi < suffixes.len) { + assert(same(readfile(strings.concat(works[0], "/", actions[ai], + suffixes[xi])), readfile(strings.concat(works[1], "/", + actions[ai], suffixes[xi])))); + xi += 1; + }; + ai += 1; + }; + i += 1; + }; + let supportactions: []str = ["ascii", "bytes", "encoding.utf8", + "fnmatch", "os", "rt", "strings", "test", "time", "types"]; + let ai: i32 = 0; + for (ai < supportactions.len) { + i = 0; + for (i < suffixes.len) { + assert(same(readfile(strings.concat(works[0], "/", + supportactions[ai], suffixes[i])), readfile(strings.concat(works[1], + "/", supportactions[ai], suffixes[i])))); + i += 1; + }; + ai += 1; + }; + assert(same(readfile(strings.concat(root, "/c-same-p051")), + readfile(strings.concat(root, "/ww-same-p051")))); + assert(same(readfile(strings.concat(root, "/c-external-p051")), + readfile(strings.concat(root, "/ww-external-p051")))); + assert(same(readfile(strings.concat(root, "/c-alias-p204")), + readfile(strings.concat(root, "/ww-alias-p204")))); + let internalunit: str = readfile(strings.concat(works[0], + "/p051-internal-test.unit.ww")); + let externalunit: str = readfile(strings.concat(works[0], + "/p051_test-external-test.unit.ww")); + let productionunit: str = readfile(strings.concat(works[0], + "/p051.unit.ww")); + assert(has(internalunit, "DYNAMIC_PRODUCTION_p051")); + assert(has(internalunit, "DYNAMIC_INTERNAL_p051")); + assert(!has(internalunit, "DYNAMIC_EXTERNAL_p051")); + assert(has(externalunit, "DYNAMIC_EXTERNAL_p051")); + assert(!has(externalunit, "DYNAMIC_PRODUCTION_p051")); + assert(has(productionunit, "DYNAMIC_PRODUCTION_p051")); + assert(!has(productionunit, "DYNAMIC_INTERNAL_p051")); + assert(!has(productionunit, "DYNAMIC_EXTERNAL_p051")); clean(root); }; @@ -4136,8 +4874,8 @@ fn runtimepath(relative: str) str = { assert(has(ctraced, strings.concat(" -L ", root, " -l extra"))); // Keep this row on default sibling discovery: a copied driver beside a - // tracing linker and exact compiler/assembler siblings exposes WWstage's - // joined native-linker flag contract independently of tool overrides. + // tracing linker and exact compiler/assembler siblings proves WWstage uses + // the same separate flag/value linker argv contract as Cstage. let tracedir: str = strings.concat(repo(), "/out/link-flag-trace-", workescape(root)); assert(os.mkdir(tracedir, 448i32) == 0); @@ -4168,7 +4906,10 @@ fn runtimepath(relative: str) str = { (60i64 * (time.second: i64)): time.duration, &out); expectexit(&out, 0); let wtraced: str = readfile(linktrace); - assert(has(wtraced, strings.concat(" -L", root, " -lextra"))); + let wwdelta: str = strings.sub(wtraced, ctraced.len, wtraced.len); + assert(has(wwdelta, strings.concat(" -L ", root, " -l extra"))); + assert(!has(wwdelta, strings.concat(" -L", root))); + assert(!has(wwdelta, " -lextra")); assert(same(readfile(cbin), readfile(wwbin))); let crun: []str = [cbin]; let wrun: []str = [wwbin];