diff --git a/docs/build-system.md b/docs/build-system.md index 55fe3a35..6c39e5f6 100644 --- a/docs/build-system.md +++ b/docs/build-system.md @@ -3855,6 +3855,139 @@ preloader, parallel action scheduler, or network behavior. Normal local WW builds and tests remain offline, manifest-free, registry-free, database-free, CAS-free, and network-free. +### 11.15 Implemented internal-package import visibility + +A source import whose complete canonical import path contains a directory +component named exactly `internal` is now contextual. Locate the final such +component. Its parent directory is the ownership boundary, and the import is +legal only when the importing source package's canonical filesystem directory +is that boundary or a descendant at a real path-component boundary. Thus +`domain.client` may import `domain.internal.secret`, while `outsider` and +`domainx.client` may not. `internalx` has no special meaning. For +`domain.internal.outer.internal.deep`, the final `internal` wins and the owner +is `domain.internal.outer`, not `domain`. + +The complete dotted import identity determines whether the rule applies and +how many components comprise the final `internal` plus its following suffix. +WW removes those components from that source edge's resolved lexical target +route, canonicalizes the resulting owner directory physically, and compares it +with the already canonical physical importer directory. Stripping precedes +physical canonicalization because a symlink at or below `internal` may point to +a target with a different name or depth; an action spelling interned by an +earlier edge is never reused for this contextual calculation. Cstage obtains +directory canonicalization through `realpath`; WWstage uses its equivalent +`chdir`/`getcwd` canonicalization. For explicit single-file compatibility +roots, Cstage uses `realpath` and WWstage's component walker uses +`lstat`/`readlink` plus the same canonical current-directory representation; +both preserve component and trailing-directory semantics across symlinks. The +equality-or-`/`-boundary comparison never uses a raw string prefix. This +division is intentional: an unrelated physical +ancestor literally named `internal` does not impose visibility on an import +whose canonical identity has no such component, while a symlink spelling at or +below an import-path `internal` cannot move the effective owner or importer. +Explicit single-file compatibility roots use their canonicalized containing +directory as the importer context. Directly selecting an internal directory as +a command root remains legal because selection is not a source import. + +Visibility is an import-edge property, not part of canonical package or action +identity. Both drivers resolve the complete target, intern or reuse its one +canonical production action, and then perform the importer-context check before +accepting the source binding or dependency edge. Consequently an allowed +importer may load and compile the target normally, but that cached action cannot +authorize a later forbidden importer. Reversing requested products or visiting +the forbidden importer first produces the same result. No declared package +name, leaf, output name, product ordinal, storage locator, hash, or test variant +participates in the decision. Legal edges therefore retain the existing +owner-only byte-sorted unit, exact sorted/deduplicated direct `.wwi` compiler +inputs, deterministic archive, and complete archive-only linker closure. + +Cstage represents the rule with the bounded +`sep_internal_parent_count`/`sep_internal_import_allowed` helpers in +`cmd/ww/main.c`. WWstage has the isomorphic +`sepinternalparentcount`/`sepinternalimportallowed` helpers in +`selfhost/cmd/ww/main.ww`. Neither adds a package field, fixed-size package +table, second action universe, or action-key input. A distinct +`SEP_LOAD_INTERNAL` result propagates through the iterative loader. On rejection +both stages emit the importing parser position followed by exactly: + +```text +use of internal package not allowed +``` + +The command returns immediately from graph loading, before directory-identity +finalization, unit composition, workdir owner validation, stale-voucher +invalidation, tool-identity recording, compiler, assembler, in-driver archive +production, linker, publication, or execution. A cold rejection therefore +commits no target/importer artifact or `.wwtool.*` state. A forbidden warm +request cannot use a previously committed target to bypass the check and leaves +the already committed target voucher and all tool records byte-unchanged. +Allocation or path canonicalization failure remains a deterministic +command-fatal pre-tool error; the visibility helper does not change stable +package indices or growth behavior. + +Every selected source file reaches the same driver scan. The rule therefore +applies to ordinary production sources, same-package test sources in the +internal production-plus-test variant, external `_test` sources, and real +source imports inside the test-support package. Generated-main-to-variant, +generated-main-to-support, and coordinator product wiring remain synthetic +edges and are not retroactively treated as source imports. No coordinator +change is required: `cmd/wwtest` only dispatches, and +`internal/wwpackage/package.ww` only discovers and classifies source groups, +constructs the union request, and runs the driver; neither resolves a source +import or owns its importing position. + +This follows only the pinned official Go 1.26.5 source at commit +`c19862e5f8415b4f24b189d065ed739517c548ba`: + +- `loadImport` resolves and reuses the canonical package first, then explicitly + checks the rule on every import because the result depends on the importing + code, and attaches the importing position + ([`pkg.go`, lines 787–791](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L787-L791)). +- The rule is the tree rooted at the parent of the target's `internal` + directory + ([`pkg.go`, lines 1463–1471](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1463-L1471)). + A package named directly on the command line is not an import + ([lines 1498–1502](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1498-L1502)), + and the import-path boundary is located before filesystem containment + ([lines 1505–1515](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1505-L1515)). +- Go's filesystem branch cleans the importer and owner, requires a + path-component-aware prefix, and retries with both paths symlink-expanded + ([`pkg.go`, lines 1534–1546](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1534-L1546)). + Its rejection text is exactly the diagnostic above + ([lines 1564–1571](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1564-L1571)), + and its exact-component search deliberately selects the final `internal` + element as the most restrictive rule + ([lines 1574–1590](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1574-L1590)). +- Same-package and external test imports both pass through ordinary + `loadImport` with their own source positions before variant construction + ([`test.go`, lines 102–161](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L102-L161)). + Generated test-main dependencies are synthesized separately + ([lines 307–330](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L307-L330)). +- Go's downstream action cache keys canonical package actions independently + and consumes the already validated direct package imports, so visibility does + not belong in action identity + ([`action.go`, lines 437–455](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L437-L455), + [lines 628–657](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L628-L657)). + +The focused native observer `internal_package_import_visibility` generates its +lexical package/import tree under a temporary physical ancestor also named +`internal`, while deliberate symlink destinations live outside that ancestor, +so that unrelated-host-path case is exercised rather than documented only. It +builds and runs allowed descendant, nested-final, `internalx`, and symlinked +physical-owner cases; rejects outsider, sibling-prefix, nested-final, and +symlink-escape cases at exact source positions with empty tool traces; maps an +internal target symlinked to a differently shaped physical path back to its +lexical owner's canonical directory; builds an internal package directly; and +reverses requested allowed/forbidden product-descriptor order around one reused +target. It checks ordinary, internal-test, and external-test +source imports, generated-main transitive-export isolation, owner-only source +order, direct export inputs, archive-only link closure, runtime results, warm +no-op package production, rejection-state preservation, and the primary +production chain's unit, export, assembly, object, archive, binary, and exact +normalized tool arguments across independent cold Cstage and WWstage work +roots. Test variants additionally compare their unit/export/archive bytes and +exercise byte-equivalent runtime output in both stages. + ## 12. Candidate architectures and hard-gate decision Five candidates were developed as coherent systems, not as feature bins. diff --git a/test/package/package_test.ww b/test/package/package_test.ww index 0508e4e9..3dd1b8e0 100644 --- a/test/package/package_test.ww +++ b/test/package/package_test.ww @@ -423,6 +423,18 @@ fn mkdirall(path: str) void = { }; }; +fn normalizedtrace(trace: str, work: str, output: str) str = { + let first: str; + match (strings.replace(trace, work, "$WORK/")) { + case let value: str => first = value; + case nomem => abort("trace normalization failed"); + }; + match (strings.replace(first, output, "$OUTPUT")) { + case let value: str => return value; + case nomem => abort("trace normalization failed"); + }; +}; + @test fn deterministic_routing_and_filters() void = { let root: str = fresh(); let target: str = packagepath("routing"); @@ -3550,6 +3562,617 @@ fn mkdirall(path: str) void = { clean(root); }; +@test fn internal_package_import_visibility() void = { + let root: str = fresh(); + let source: str = strings.concat(root, "/host/internal/source"); + let runtime: str = strings.concat(root, "/runtime"); + let tools: str = strings.concat(root, "/tools"); + let domain: str = strings.concat(source, "/domain"); + let leaf: str = strings.concat(domain, "/leaf"); + let secret: str = strings.concat(domain, "/internal/secret"); + let client: str = strings.concat(domain, "/client"); + let outsider: str = strings.concat(source, "/outsider"); + let prefixclient: str = strings.concat(source, "/domainx/client"); + let deep: str = strings.concat(domain, + "/internal/outer/internal/deep"); + let deepclient: str = strings.concat(domain, "/internal/outer/client"); + let nestedoutsider: str = strings.concat(domain, "/nestedclient"); + let internalx: str = strings.concat(domain, "/internalx/visible"); + let internalxclient: str = strings.concat(source, "/internalxclient"); + let testclient: str = strings.concat(domain, "/testclient"); + let outsame: str = strings.concat(source, "/outsame"); + let outexternal: str = strings.concat(source, "/outexternal"); + let physical: str = strings.concat(root, "/physical/domain"); + let linkedsecret: str = strings.concat(physical, "/internal/secret"); + let linkedclient: str = strings.concat(physical, "/client"); + let rawallowed: str = strings.concat(source, "/alias/rawclient.ww"); + let rawalias: str = strings.concat(domain, "/rawalias.ww"); + let rawescape: str = strings.concat(domain, "/rawescape.ww"); + let aliasinternal: str = strings.concat(source, "/alias/internal"); + let aliasclient: str = strings.concat(source, "/alias/client"); + let aliasownedclient: str = strings.concat(source, + "/alias/owned-client"); + let aliasoutsider: str = strings.concat(source, "/alias/outsider"); + let shapedomain: str = strings.concat(source, "/shape/domain"); + let shapeinternal: str = strings.concat(shapedomain, "/internal"); + let shapeclient: str = strings.concat(shapedomain, "/client"); + let shapeoutsider: str = strings.concat(source, "/shape/outsider"); + let shapevault: str = strings.concat(root, "/vault"); + mkdirall(leaf); mkdirall(secret); mkdirall(client); mkdirall(outsider); + mkdirall(prefixclient); mkdirall(deep); mkdirall(deepclient); + mkdirall(nestedoutsider); mkdirall(internalx); mkdirall(internalxclient); + mkdirall(testclient); mkdirall(outsame); mkdirall(outexternal); + mkdirall(linkedsecret); mkdirall(linkedclient); mkdirall(aliasinternal); + mkdirall(aliasownedclient); mkdirall(shapeinternal); mkdirall(shapeclient); + mkdirall(shapeoutsider); mkdirall(shapevault); mkdirall(runtime); + mkdirall(tools); + writefile(strings.concat(runtime, "/libwwrt.a"), + readfile(strings.concat(repo(), "/out/lib/libwwrt.a"))); + + let leafbody: str = strings.concat( + "package leaf;\n", + "export fn value() i32 = { return 20; };\n"); + let secreta: str = strings.concat( + "package secret;\n", + "import domain.leaf;\n", + "export fn value() i32 = { return leaf.value() + 22; };\n"); + let secretz: str = strings.concat( + "package secret;\n", + "fn private_value() i32 = { return 99; };\n"); + let clientbody: str = strings.concat( + "package main;\n", + "import domain.internal.secret;\n", + "import domain.internal.secret;\n", + "fn main() i32 = { return secret.value(); };\n"); + writefile(strings.concat(leaf, "/leaf.ww"), leafbody); + writefile(strings.concat(secret, "/a.ww"), secreta); + writefile(strings.concat(secret, "/z.ww"), secretz); + writefile(strings.concat(client, "/main.ww"), clientbody); + writefile(strings.concat(outsider, "/main.ww"), strings.concat( + "package main;\nimport domain.internal.secret;\n", + "fn main() i32 = { return secret.value(); };\n")); + writefile(strings.concat(prefixclient, "/main.ww"), strings.concat( + "package main;\nimport domain.internal.secret;\n", + "fn main() i32 = { return secret.value(); };\n")); + writefile(strings.concat(deep, "/deep.ww"), strings.concat( + "package deep;\n", + "export fn value() i32 = { return 7; };\n")); + writefile(strings.concat(deepclient, "/main.ww"), strings.concat( + "package main;\n", + "import domain.internal.outer.internal.deep;\n", + "fn main() i32 = { return deep.value(); };\n")); + writefile(strings.concat(nestedoutsider, "/main.ww"), strings.concat( + "package main;\n", + "import domain.internal.outer.internal.deep;\n", + "fn main() i32 = { return deep.value(); };\n")); + writefile(strings.concat(internalx, "/visible.ww"), strings.concat( + "package visible;\n", + "export fn value() i32 = { return 9; };\n")); + writefile(strings.concat(internalxclient, "/main.ww"), strings.concat( + "package main;\nimport domain.internalx.visible;\n", + "fn main() i32 = { return visible.value(); };\n")); + + writefile(strings.concat(linkedsecret, "/secret.ww"), strings.concat( + "package secret;\n", + "export fn value() i32 = { return 31; };\n")); + writefile(strings.concat(linkedclient, "/main.ww"), strings.concat( + "package main;\nimport alias.internal.secret;\n", + "fn main() i32 = { return secret.value(); };\n")); + writefile(rawallowed, strings.concat( + "package main;\nimport alias.internal.secret;\n", + "fn main() i32 = { return secret.value(); };\n")); + assert(os.symlink(rawallowed, rawalias) == 0); + assert(os.symlink(strings.concat(outsider, "/main.ww"), rawescape) == 0); + assert(os.symlink(linkedsecret, + strings.concat(aliasinternal, "/secret")) == 0); + writefile(strings.concat(aliasownedclient, "/main.ww"), strings.concat( + "package main;\nimport alias.internal.secret;\n", + "fn main() i32 = { return secret.value(); };\n")); + assert(os.symlink(aliasownedclient, aliasclient) == 0); + assert(os.symlink(linkedclient, aliasoutsider) == 0); + writefile(strings.concat(shapevault, "/secret.ww"), strings.concat( + "package secret;\n", + "export fn value() i32 = { return 17; };\n")); + assert(os.symlink(shapevault, + strings.concat(shapeinternal, "/secret")) == 0); + writefile(strings.concat(shapeclient, "/main.ww"), strings.concat( + "package main;\nimport shape.domain.internal.secret;\n", + "fn main() i32 = { return secret.value(); };\n")); + writefile(strings.concat(shapeoutsider, "/main.ww"), strings.concat( + "package main;\nimport shape.domain.internal.secret;\n", + "fn main() i32 = { return secret.value(); };\n")); + + writefile(strings.concat(testclient, "/testclient.ww"), strings.concat( + "package testclient;\n", + "export fn value() i32 = { return 1; };\n")); + writefile(strings.concat(testclient, "/same_test.ww"), strings.concat( + "package testclient;\nimport domain.internal.secret;\n", + "@test fn allowed_same() void = {\n", + " assert(secret.value() == 42);\n};\n")); + writefile(strings.concat(testclient, "/external_test.ww"), strings.concat( + "package testclient_test;\nimport domain.internal.secret;\n", + "import domain.testclient;\n", + "@test fn allowed_external() void = {\n", + " assert(secret.value() + testclient.value() == 43);\n};\n")); + writefile(strings.concat(outsame, "/outsame.ww"), + "package outsame;\nexport fn value() i32 = { return 1; };\n"); + writefile(strings.concat(outsame, "/same_test.ww"), strings.concat( + "package outsame;\nimport domain.internal.secret;\n", + "@test fn forbidden_same() void = { assert(false); };\n")); + writefile(strings.concat(outexternal, "/outexternal.ww"), + "package outexternal;\nexport fn value() i32 = { return 1; };\n"); + writefile(strings.concat(outexternal, "/external_test.ww"), strings.concat( + "package outexternal_test;\nimport domain.internal.secret;\n", + "@test fn forbidden_external() void = { assert(false); };\n")); + + let compilerwrapper: str = strings.concat(tools, "/w6c.sh"); + let assemblerwrapper: str = strings.concat(tools, "/w6a.sh"); + let linkerwrapper: str = strings.concat(tools, "/w6l.sh"); + writeexecutable(compilerwrapper, strings.concat( + "#!/bin/sh\nprintf 'BEGIN' >> \"$WW_INTERNAL_COMPILER_TRACE\"\n", + "for arg in \"$@\"; do printf '<%s>' \"$arg\" >> ", + "\"$WW_INTERNAL_COMPILER_TRACE\"; done\n", + "printf '\\n' >> \"$WW_INTERNAL_COMPILER_TRACE\"\n", + "exec \"$WW_INTERNAL_REAL_COMPILER\" \"$@\"\n")); + writeexecutable(assemblerwrapper, strings.concat( + "#!/bin/sh\nprintf 'BEGIN' >> \"$WW_INTERNAL_ASSEMBLER_TRACE\"\n", + "for arg in \"$@\"; do printf '<%s>' \"$arg\" >> ", + "\"$WW_INTERNAL_ASSEMBLER_TRACE\"; done\n", + "printf '\\n' >> \"$WW_INTERNAL_ASSEMBLER_TRACE\"\n", + "exec \"$WW_INTERNAL_REAL_ASSEMBLER\" \"$@\"\n")); + writeexecutable(linkerwrapper, strings.concat( + "#!/bin/sh\nprintf 'BEGIN' >> \"$WW_INTERNAL_LINKER_TRACE\"\n", + "for arg in \"$@\"; do printf '<%s>' \"$arg\" >> ", + "\"$WW_INTERNAL_LINKER_TRACE\"; done\n", + "printf '\\n' >> \"$WW_INTERNAL_LINKER_TRACE\"\n", + "exec \"$WW_INTERNAL_REAL_LINKER\" \"$@\"\n")); + + let stages: []str = ["ww", "ww_ww"]; + let compilers: []str = ["w6c", "w6c_ww"]; + let assemblers: []str = ["w6a", "w6a_ww"]; + let linkers: []str = ["w6l", "w6l_ww"]; + let labels: []str = ["0", "1", "2", "3", "4"]; + let works: []str = [strings.concat(root, "/c-work"), + strings.concat(root, "/ww-work")]; + let bins: []str = [strings.concat(root, "/c-bin"), + strings.concat(root, "/ww-bin")]; + let compilertraces: []str = [strings.concat(root, "/c-compiler"), + strings.concat(root, "/ww-compiler")]; + let assemblertraces: []str = [strings.concat(root, "/c-assembler"), + strings.concat(root, "/ww-assembler")]; + let linkertraces: []str = [strings.concat(root, "/c-linker"), + strings.concat(root, "/ww-linker")]; + let artifacts: []str = ["domain.leaf", "domain.internal.secret", + "domain.client"]; + let suffixes: []str = [".unit.ww", ".wwi", ".s", ".o", ".a"]; + let references: []str = alloc([], (artifacts.len * suffixes.len): u64)!; + let ri: i32 = 0; + for (ri < artifacts.len * suffixes.len) { + append(references, ""); ri += 1; + }; + let referencebin: str = ""; + let referencecompiler: str = ""; + let referenceassembler: str = ""; + let referencelinker: str = ""; + let referencediagnostic: str = ""; + let referencetestout: str = ""; + let referencetesterr: str = ""; + let referencetestartifacts: []str = alloc([], 12u64)!; + ri = 0; + for (ri < 12) { append(referencetestartifacts, ""); ri += 1; }; + let allowedreferences: []str = ["", "", "", ""]; + let testrejectreferences: []str = ["", ""]; + let orderreference: str = ""; + let baseenv: []str = os.getenvs(); + let si: i32 = 0; + for (si < stages.len) { + assert(os.mkdir(works[si], 448i32) == 0); + writefile(compilertraces[si], ""); writefile(assemblertraces[si], ""); + writefile(linkertraces[si], ""); + let env: []str = alloc([], (baseenv.len + 11): u64)!; + let ei: i32 = 0; + for (ei < baseenv.len) { + if (!strings.hasprefix(baseenv[ei], "WW_W6C=") + && !strings.hasprefix(baseenv[ei], "WW_W6A=") + && !strings.hasprefix(baseenv[ei], "WW_W6L=") + && !strings.hasprefix(baseenv[ei], + "WW_INTERNAL_COMPILER_TRACE=") + && !strings.hasprefix(baseenv[ei], + "WW_INTERNAL_ASSEMBLER_TRACE=") + && !strings.hasprefix(baseenv[ei], + "WW_INTERNAL_LINKER_TRACE=") + && !strings.hasprefix(baseenv[ei], + "WW_INTERNAL_REAL_COMPILER=") + && !strings.hasprefix(baseenv[ei], + "WW_INTERNAL_REAL_ASSEMBLER=") + && !strings.hasprefix(baseenv[ei], + "WW_INTERNAL_REAL_LINKER=") + && !strings.hasprefix(baseenv[ei], "WW_LIB=")) { + append(env, baseenv[ei]); + }; + ei += 1; + }; + append(env, strings.concat("WW_W6C=", compilerwrapper)); + append(env, strings.concat("WW_W6A=", assemblerwrapper)); + append(env, strings.concat("WW_W6L=", linkerwrapper)); + append(env, strings.concat("WW_INTERNAL_COMPILER_TRACE=", + compilertraces[si])); + append(env, strings.concat("WW_INTERNAL_ASSEMBLER_TRACE=", + assemblertraces[si])); + append(env, strings.concat("WW_INTERNAL_LINKER_TRACE=", + linkertraces[si])); + append(env, strings.concat("WW_INTERNAL_REAL_COMPILER=", + driver(compilers[si]))); + append(env, strings.concat("WW_INTERNAL_REAL_ASSEMBLER=", + driver(assemblers[si]))); + append(env, strings.concat("WW_INTERNAL_REAL_LINKER=", + driver(linkers[si]))); + append(env, strings.concat("WW_LIB=", runtime)); + + let av: []str = [driver(stages[si]), "build", "-w", works[si], + "-I", source, "-o", bins[si], client]; + let out: commandout; + runcommandenv(root, strings.concat("internal-cold-", stages[si]), av, + env, (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stdout.len == 0 && out.stderr.len == 0); + let wantsecret: str = strings.concat( + "//ww:module-reset domain.internal.secret\n", secreta, + "\n//ww:module-reset domain.internal.secret\n", secretz, "\n"); + assert(same(readfile(strings.concat(works[si], + "/domain.internal.secret.unit.ww")), wantsecret)); + let ai: i32 = 0; + for (ai < artifacts.len) { + let xi: i32 = 0; + for (xi < suffixes.len) { + let bytes: str = readfile(strings.concat(works[si], "/", + artifacts[ai], suffixes[xi])); + let index: i32 = ai * suffixes.len + xi; + if (si == 0) { references[index] = strings.dup(bytes); } + else { assert(same(references[index], bytes)); }; + xi += 1; + }; + ai += 1; + }; + let ctrace: str = readfile(compilertraces[si]); + let atrace: str = readfile(assemblertraces[si]); + let ltrace: str = readfile(linkertraces[si]); + assert(occurrences(ctrace, "\n") == 3); + assert(occurrences(atrace, "\n") == 3); + assert(occurrences(ltrace, "\n") == 1); + let secretline: str = linecontaining(ctrace, + "domain.internal.secret.unit.new"); + assert(occurrences(secretline, "domain.leaf.wwi") == 1); + assert(!has(secretline, "domain.client.wwi")); + let clientline: str = linecontaining(ctrace, + "domain.client.unit.new"); + assert(occurrences(clientline, + "domain.internal.secret.wwi") == 1); + assert(!has(clientline, "domain.leaf.wwi")); + assert(occurrences(ltrace, strings.concat(works[si], + "/domain.client.a")) == 1); + assert(occurrences(ltrace, strings.concat(works[si], + "/domain.internal.secret.a")) == 1); + assert(occurrences(ltrace, strings.concat(works[si], + "/domain.leaf.a")) == 1); + assert(!has(ltrace, ".wwi")); + let normalizedcompiler: str = normalizedtrace(ctrace, + strings.concat(works[si], "/"), bins[si]); + let normalizedassembler: str = normalizedtrace(atrace, + strings.concat(works[si], "/"), bins[si]); + let normalizedlinker: str = normalizedtrace(ltrace, + strings.concat(works[si], "/"), bins[si]); + if (si == 0) { + referencecompiler = strings.dup(normalizedcompiler); + referenceassembler = strings.dup(normalizedassembler); + referencelinker = strings.dup(normalizedlinker); + referencebin = strings.dup(readfile(bins[si])); + } else { + assert(same(referencecompiler, normalizedcompiler)); + assert(same(referenceassembler, normalizedassembler)); + assert(same(referencelinker, normalizedlinker)); + assert(same(referencebin, readfile(bins[si]))); + }; + let runav: []str = [bins[si]]; + runcommand(root, strings.concat("internal-run-", stages[si]), runav, + (60i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 42); + assert(out.stdout.len == 0 && out.stderr.len == 0); + + let voucher: str = readfile(strings.concat(works[si], + "/domain.internal.secret.unit.ww")); + let toolww: str = readfile(strings.concat(works[si], "/.wwtool.ww")); + let toolc: str = readfile(strings.concat(works[si], "/.wwtool.w6c")); + let toola: str = readfile(strings.concat(works[si], "/.wwtool.w6a")); + let toolstamp: str = readfile(strings.concat(works[si], + "/.wwtool.stamp")); + rewritefile(compilertraces[si], ""); rewritefile(assemblertraces[si], ""); + rewritefile(linkertraces[si], ""); + runcommandenv(root, strings.concat("internal-warm-", stages[si]), av, + env, (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(readfile(compilertraces[si]).len == 0); + assert(readfile(assemblertraces[si]).len == 0); + assert(occurrences(readfile(linkertraces[si]), "\n") == 1); + assert(same(voucher, readfile(strings.concat(works[si], + "/domain.internal.secret.unit.ww")))); + + rewritefile(compilertraces[si], ""); rewritefile(assemblertraces[si], ""); + rewritefile(linkertraces[si], ""); + let badout: str = strings.concat(root, "/warm-forbidden-", stages[si]); + let badav: []str = [driver(stages[si]), "build", "-w", works[si], + "-I", source, "-o", badout, outsider]; + runcommandenv(root, strings.concat("internal-warm-forbidden-", + stages[si]), badav, env, + (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 1); + let wantdiag: str = strings.concat(outsider, + "/main.ww:2:1: error: use of internal package ", + "domain.internal.secret not allowed\n"); + assert(same(out.stderr, wantdiag)); + if (si == 0) { referencediagnostic = strings.dup(out.stderr); } + else { assert(same(referencediagnostic, out.stderr)); }; + assert(readfile(compilertraces[si]).len == 0); + assert(readfile(assemblertraces[si]).len == 0); + assert(readfile(linkertraces[si]).len == 0); + assert(!os.exists(badout)); + assert(!os.exists(strings.concat(works[si], "/outsider.unit.ww"))); + assert(same(voucher, readfile(strings.concat(works[si], + "/domain.internal.secret.unit.ww")))); + assert(same(toolww, readfile(strings.concat(works[si], "/.wwtool.ww")))); + assert(same(toolc, readfile(strings.concat(works[si], "/.wwtool.w6c")))); + assert(same(toola, readfile(strings.concat(works[si], "/.wwtool.w6a")))); + assert(same(toolstamp, readfile(strings.concat(works[si], + "/.wwtool.stamp")))); + + let coldreject: str = strings.concat(root, "/cold-reject-", stages[si]); + assert(os.mkdir(coldreject, 448i32) == 0); + rewritefile(compilertraces[si], ""); rewritefile(assemblertraces[si], ""); + rewritefile(linkertraces[si], ""); + let coldav: []str = [driver(stages[si]), "build", "-w", coldreject, + "-I", source, "-o", strings.concat(coldreject, ".bin"), outsider]; + runcommandenv(root, strings.concat("internal-cold-forbidden-", + stages[si]), coldav, env, + (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 1); + assert(same(out.stderr, wantdiag)); + assert(readfile(compilertraces[si]).len == 0); + assert(readfile(assemblertraces[si]).len == 0); + assert(readfile(linkertraces[si]).len == 0); + assert(!os.exists(strings.concat(coldreject, "/.wwtool.ww"))); + assert(!os.exists(strings.concat(coldreject, "/.wwtool.w6c"))); + assert(!os.exists(strings.concat(coldreject, "/.wwtool.w6a"))); + assert(!os.exists(strings.concat(coldreject, "/.wwtool.stamp"))); + assert(!os.exists(strings.concat(coldreject, ".bin"))); + assert(!os.exists(strings.concat(coldreject, + "/domain.internal.secret.unit.ww"))); + assert(!os.exists(strings.concat(coldreject, + "/domain.internal.secret.a"))); + assert(!os.exists(strings.concat(coldreject, "/outsider.unit.ww"))); + + let allowedtargets: []str = [deepclient, internalxclient, aliasclient, + shapeclient]; + let allowedcodes: []i32 = [7, 9, 31, 17]; + let allowedi: i32 = 0; + for (allowedi < allowedtargets.len) { + let allowedout: str = strings.concat(root, "/allowed-", stages[si], + "-", labels[allowedi]); + let allowedav: []str = [driver(stages[si]), "build", "-I", source, + "-o", allowedout, allowedtargets[allowedi]]; + runcommandenv(root, strings.concat("internal-allowed-", stages[si], + "-", labels[allowedi]), allowedav, env, + (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + let allowedrun: []str = [allowedout]; + runcommand(root, strings.concat("internal-allowed-run-", stages[si], + "-", labels[allowedi]), allowedrun, + (60i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, allowedcodes[allowedi]); + if (si == 0) { + allowedreferences[allowedi] = strings.dup(readfile(allowedout)); + } else { + assert(same(allowedreferences[allowedi], readfile(allowedout))); + }; + allowedi += 1; + }; + let rawout: str = strings.concat(root, "/raw-allowed-", stages[si]); + let rawav: []str = [driver(stages[si]), "build", "-I", source, + "-o", rawout, rawalias]; + runcommandenv(root, strings.concat("internal-raw-allowed-", stages[si]), + rawav, env, (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + let rawrun: []str = [rawout]; + runcommand(root, strings.concat("internal-raw-run-", stages[si]), rawrun, + (60i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 31); + + let directout: str = strings.concat(root, "/direct-", stages[si], ".a"); + let directav: []str = [driver(stages[si]), "build", "-I", source, + "-o", directout, secret]; + runcommandenv(root, strings.concat("internal-direct-", stages[si]), + directav, env, (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(same(references[1 * suffixes.len + 4], readfile(directout))); + assert(same(references[1 * suffixes.len + 1], + readfile(strings.concat(directout, ".wwi")))); + + let rejecttargets: []str = [prefixclient, nestedoutsider, aliasoutsider, + rawescape, shapeoutsider]; + let rejectimports: []str = ["domain.internal.secret", + "domain.internal.outer.internal.deep", "alias.internal.secret", + "domain.internal.secret", "shape.domain.internal.secret"]; + let rejecti: i32 = 0; + for (rejecti < rejecttargets.len) { + let rejectwork: str = strings.concat(root, "/reject-", stages[si], + "-", labels[rejecti]); + assert(os.mkdir(rejectwork, 448i32) == 0); + rewritefile(compilertraces[si], ""); + rewritefile(assemblertraces[si], ""); + rewritefile(linkertraces[si], ""); + let rejectav: []str = [driver(stages[si]), "build", "-w", + rejectwork, "-I", source, "-o", strings.concat(rejectwork, + ".bin"), rejecttargets[rejecti]]; + runcommandenv(root, strings.concat("internal-reject-", stages[si], + "-", labels[rejecti]), rejectav, env, + (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 1); + let rejectsource: str = strings.concat(rejecttargets[rejecti], + "/main.ww"); + if (rejecti == 3) { rejectsource = rawescape; }; + let rejectdiag: str = strings.concat(rejectsource, + ":2:1: error: use of internal package ", + rejectimports[rejecti], " not allowed\n"); + assert(same(out.stderr, rejectdiag)); + assert(readfile(compilertraces[si]).len == 0); + assert(readfile(assemblertraces[si]).len == 0); + assert(readfile(linkertraces[si]).len == 0); + assert(!os.exists(strings.concat(rejectwork, "/.wwtool.ww"))); + assert(!os.exists(strings.concat(rejectwork, "/.wwtool.w6c"))); + assert(!os.exists(strings.concat(rejectwork, "/.wwtool.w6a"))); + assert(!os.exists(strings.concat(rejectwork, "/.wwtool.stamp"))); + assert(!os.exists(strings.concat(rejectwork, ".bin"))); + rejecti += 1; + }; + + let testworkroot: str = strings.concat(root, "/test-work-", stages[si]); + assert(os.mkdir(testworkroot, 448i32) == 0); + rewritefile(compilertraces[si], ""); rewritefile(assemblertraces[si], ""); + rewritefile(linkertraces[si], ""); + let testav: []str = [driver(stages[si]), "test", "-w", testworkroot, + "-I", source, testclient]; + runcommandenv(root, strings.concat("internal-test-allowed-", stages[si]), + testav, env, (180i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(has(out.stdout, "allowed_same ... ok\n")); + assert(has(out.stdout, "allowed_external ... ok\n")); + if (si == 0) { + referencetestout = strings.dup(out.stdout); + referencetesterr = strings.dup(out.stderr); + } else { + assert(same(referencetestout, out.stdout)); + assert(same(referencetesterr, out.stderr)); + }; + let testwork: str = strings.concat(testworkroot, "/", + requestworkkey(testclient), "/"); + let testactions: []str = ["domain.testclient-internal-test", + "domain.testclient_test-external-test", + "domain.testclient-internal-test-main", + "domain.testclient_test-external-test-main"]; + ai = 0; + for (ai < testactions.len) { + let testsuffixes: []str = [".unit.ww", ".wwi", ".a"]; + let xi: i32 = 0; + for (xi < testsuffixes.len) { + let bytes: str = readfile(strings.concat(testwork, + testactions[ai], testsuffixes[xi])); + let index: i32 = ai * testsuffixes.len + xi; + if (si == 0) { + referencetestartifacts[index] = strings.dup(bytes); + } else { + assert(same(referencetestartifacts[index], bytes)); + }; + xi += 1; + }; + ai += 1; + }; + let testcompiletrace: str = readfile(compilertraces[si]); + let sameline: str = linecontaining(testcompiletrace, + "domain.testclient-internal-test.unit"); + let externalline: str = linecontaining(testcompiletrace, + "domain.testclient_test-external-test.unit"); + assert(occurrences(sameline, "domain.internal.secret.wwi") == 1); + assert(occurrences(externalline, + "domain.internal.secret.wwi") == 1); + assert(!has(sameline, "domain.leaf.wwi")); + assert(!has(externalline, "domain.leaf.wwi")); + let samemainline: str = linecontaining(testcompiletrace, + "domain.testclient-internal-test-main.unit"); + let externalmainline: str = linecontaining(testcompiletrace, + "domain.testclient_test-external-test-main.unit"); + assert(!has(samemainline, "domain.internal.secret.wwi")); + assert(!has(externalmainline, "domain.internal.secret.wwi")); + assert(occurrences(testcompiletrace, "--test-package") == 2); + assert(occurrences(testcompiletrace, + "<-T><--entry><--test-support-module>") == 2); + assert(!has(readfile(linkertraces[si]), ".wwi")); + + let testrejects: []str = [outsame, outexternal]; + let testfiles: []str = ["/same_test.ww", "/external_test.ww"]; + let tri: i32 = 0; + for (tri < testrejects.len) { + let tw: str = strings.concat(root, "/test-reject-", stages[si], + "-", labels[tri]); + assert(os.mkdir(tw, 448i32) == 0); + rewritefile(compilertraces[si], ""); + rewritefile(assemblertraces[si], ""); + rewritefile(linkertraces[si], ""); + let tav: []str = [driver(stages[si]), "test", "-w", tw, + "-I", source, testrejects[tri]]; + runcommandenv(root, strings.concat("internal-test-reject-", + stages[si], "-", labels[tri]), tav, env, + (180i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 1); + let testdiag: str = strings.concat(testrejects[tri], testfiles[tri], + ":2:1: error: use of internal package ", + "domain.internal.secret not allowed\n"); + assert(has(out.stderr, testdiag)); + if (si == 0) { + testrejectreferences[tri] = strings.dup(out.stderr); + } else { + assert(same(testrejectreferences[tri], out.stderr)); + }; + assert(readfile(compilertraces[si]).len == 0); + assert(readfile(assemblertraces[si]).len == 0); + assert(readfile(linkertraces[si]).len == 0); + tri += 1; + }; + + let orderwork: str = strings.concat(root, "/order-work-", stages[si]); + assert(os.mkdir(orderwork, 448i32) == 0); + let orderforward: []str = [driver(stages[si]), "test", "-c", "-w", + orderwork, "-I", source, + "--ww-package-test", "production", "main", client, + strings.concat(root, "/order-client-", stages[si]), + strings.concat(root, "/order-client-", stages[si], ".status"), + "--ww-package-test", "production", "main", outsider, + strings.concat(root, "/order-outsider-", stages[si]), + strings.concat(root, "/order-outsider-", stages[si], ".status"), + client]; + let orderreverse: []str = [driver(stages[si]), "test", "-c", "-w", + orderwork, "-I", source, + "--ww-package-test", "production", "main", outsider, + strings.concat(root, "/order-outsider-", stages[si]), + strings.concat(root, "/order-outsider-", stages[si], ".status"), + "--ww-package-test", "production", "main", client, + strings.concat(root, "/order-client-", stages[si]), + strings.concat(root, "/order-client-", stages[si], ".status"), + client]; + let orderrequests: [][]str = [orderforward, orderreverse]; + let orderdiags: []str = ["", ""]; + let oi: i32 = 0; + for (oi < 2) { + rewritefile(compilertraces[si], ""); + rewritefile(assemblertraces[si], ""); + rewritefile(linkertraces[si], ""); + runcommandenv(root, strings.concat("internal-order-", stages[si], + "-", labels[oi]), orderrequests[oi], env, + (180i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 1); + orderdiags[oi] = strings.dup(out.stderr); + assert(same(out.stderr, wantdiag)); + assert(readfile(compilertraces[si]).len == 0); + assert(readfile(assemblertraces[si]).len == 0); + assert(readfile(linkertraces[si]).len == 0); + oi += 1; + }; + assert(same(orderdiags[0], orderdiags[1])); + if (si == 0) { orderreference = strings.dup(orderdiags[0]); } + else { assert(same(orderreference, orderdiags[0])); }; + si += 1; + }; + clean(root); +}; + @test fn exact_package_tool_argv_stage_parity() void = { let root: str = fresh(); let source: str = strings.concat(root, "/source tree");