Docker-Canonical Build and License Generation

Status: implemented. Revised after two rounds of review, then rebased onto origin/main and re-verified there — see §11 for what changed and why.

The cross-host generation matrix was dropped by decision, and replaced by a cheaper cross-host build-system workflow (§7.2).

Upstream tracking issue: apache/skywalking#13996 (“Introduce Docker-based build system for consistent license-file generation”), milestone BanyanDB 0.12.0. Parent chore: #13995.

Summary

Contributors on different host operating systems produce slightly different generated license artifacts — line endings, entry order, file ordering — which shows up as noisy diffs and CI churn. This design makes a pinned Linux build environment the canonical way to produce the committed license artifacts, and adds a byte-level verifier that proves the output is identical regardless of the host OS.

Four separable pieces, in dependency order:

1. Fix the checkout contract (§3) — .gitattributes currently contains a typo that leaves line-ending behaviour undefined.
2. A byte-level verifier (§4) — a path+hash manifest over the complete generated surface, comparable across runs, across hosts, and against the committed blobs. Built first, because it is what makes every other claim falsifiable.
3. A pinned build environment (§5, §6) — a digest-pinned image, and make wrappers that run the existing recipes in it against a container-side copy of the tree.
4. CI wiring (§7) — the canonical command in the ubuntu check job, with the native and container outputs compared to each other.

Out of scope, deliberately split into separate issues:

• Release-archive reproducibility (release-binary / release-source timestamps, ordering, ownership, gzip metadata). SOURCE_DATE_EPOCH is set in the image as a precondition, but scripts/release.sh does not yet implement archive normalization and this design does not attempt it.
• Making Docker canonical for every committed artifact (check, pre-push, lint, generate). Those pull in generators, linters and toolchains this image does not carry.
• Replacing the license-eye commit pin with a release tag. A SHA is already an immutable pin.

Acceptance criteria this design commits to

1. What the pipeline actually does

The embedded Vue UI in ui/ was permanently removed upstream in 0e3ee0ec (“Chore/remove embedded UI (#1380)”), so only mcp and canopy remain on the JS side. This design targets the post-removal tree.

1.1 The complete generated surface

These six groups are the only things license-dep produces, and the verifier in §4 covers exactly them:

dist/LICENSE
dist/licenses/**
mcp/LICENSE
mcp/licenses/**
canopy/LICENSE
canopy/licenses/**

The repository-root LICENSE and NOTICE are not generated by these recipes and must not be in any drift guard — a check that includes them passes vacuously and gives false confidence.

1.2 What the pinned license-eye already does

Verified against the pinned revision in the local module cache, not assumed:

• It sorts. pkg/deps/summary.go sorts dependencies by name and groups by license ID; pkg/deps/result.go sorts the printed report. Both use Go string comparison, which is locale-independent. Therefore LC_ALL has no effect on its output ordering, and a post-hoc sort over dist/licenses/ would corrupt its structure. Neither is needed.
• It installs JS dependencies itself. pkg/deps/npm.go runs npm ci and then npm prune --production as part of resolution. There is no separate install prerequisite to satisfy, in the Makefiles or in the image.
• It logs install failures instead of returning them. This matters: a failed npm ci can silently yield a partial license set that still exits zero. §4.3 covers how the verifier catches that, because it is invisible to an exit-code check.

1.3 Real sources of variance

1. Line endings in the working tree. See §3 — the current .gitattributes rule does not do what it appears to do.
2. Raw license bodies are copied verbatim. license-eye copies LICENSE text out of the Go module cache and out of node_modules. Whatever bytes are in those trees reach committed output.
3. npm and Go dependency trees differ per host. npm ci behaviour varies with the Node version and the platform; the Go module cache is platform-independent but its contents are fetched by the host’s Go.
4. Host Go version at tool-install time. go install of license-eye runs on the host.
5. Package install failures are invisible (§1.2) — a risk on every host, not a cross-OS one.

2. Why the existing guard is insufficient

2.1 It runs on one host

ci.yml runs make license-dep and make check-format on ubuntu-latest only. A contributor on Windows or macOS can produce a different diff and never find out from CI.

2.2 check-format is a modifying operation

Makefile check-format runs make format (which depends on tidy), then git add --renormalize ., then captures a diff and fails if git status -s is non-empty. It is a consistency check, not a byte-equality check: it lets Git apply its own normalization rules during staging, its captured diff can omit what it just staged, and it says nothing about a file that was generated and then deleted again.

2.3 The guard uses Git semantics where raw bytes are the requirement

git diff --exit-code compares blobs after normalization. The issue asks for byte-identical output. A verifier must hash the bytes on disk.

3. Fix the checkout contract first

.gitattributes:19 currently reads:

* text eof=lf

eof is not a Git attribute. git check-attr confirms the working tree has eof: lf and eol: unspecified — the intended LF policy is not in effect, and the typo masks the difference. The rule should be:

* text=auto eol=lf

Two changes, and only the first is the bug fix. eof=lf → eol=lf is the fix. text → text=auto is a safety improvement in its own right: the old * text forced normalization on every file, and letting Git detect binary content is safer for the unlisted binary fixtures (.seg, .snp, .tzif testdata) that nothing in the binary list covers.

Consequences, and why this is its own phase:

• eol=lf normalizes the working tree to LF on checkout on every platform, which is what the issue actually needs. It does not disable normalization during staging: that is text, which still converts a CRLF file to LF in the blob. The consequence is precise and worth stating, because it is why the LF assertion in §4.3 exists — a generated file can arrive with CRLF in the working tree while the blob it becomes is LF, so the difference is invisible to any blob comparison and visible only by reading the bytes on disk.
• Switching the rule on an existing tree can produce a large one-time renormalization diff. Measured before committing: git add --renormalize . changed nothing, because the index contained no i/crlf or i/mixed entries. The change is a pure checkout-behaviour fix, so it does not need to be quarantined into its own commit after all. The remaining risk is a contributor with a CRLF working tree, and §4.3’s LF assertion is what surfaces one.
• The existing binary patterns (*.png, *.gz, …) are unaffected.

This replaces the “scan the repo and reject CRLF” idea the first draft proposed. Rejection is not normalization: it tells a Windows contributor what broke without fixing it, and a whole-repository scan also flags files that never reach a generated artifact. The check that is kept is a targeted LF assertion over the six output groups in §4.3.

4. The verifier (built first)

4.1 scripts/ci/check/license_manifest.sh (new)

Purely native, no Docker, no network. For a given root it walks the six groups from §1.1 and emits a sorted manifest:

<sha256-of-raw-bytes>  <relative-path>

Sorted by path with LC_ALL=C. Raw bytes — no Git filters, no normalization, no text mode.

Three comparison modes:

The second mode is also what catches the silent npm ci failure of §1.2: a dependency whose license file fails to generate is a deletion relative to the committed tree, which the manifest reports.

4.2 Coverage of the modes

The manifest must compare complete filename sets, not just matching files, so that added and deleted files are both caught. The same applies to the copy-back: the wrapper removes each generated output set before extracting, because extracting over the top would keep any file the container did not produce — which is exactly what a silent npm ci failure inside license-eye leaves behind, since it exits 0. A diff of two sorted manifests does this by construction; a per-file hash loop does not, and would silently pass when a file disappears.

4.3 make check-license-outputs (new)

Runs the “against committed” comparison, plus a targeted LF assertion over the same six groups: any CR byte in a generated file is a failure, with the path printed. Wired into make check-req so it runs everywhere, including jobs that never invoke the generator — a stale committed artifact is still a stale committed artifact.

5. The build environment

5.1 Location

Everything lives under scripts/build/, never under a root build/:

• .gitignore ignores /build, and Makefile clean runs rm -rf build. A Dockerfile, an entrypoint and a pin file under root build/ would be deleted by make clean and never committed in the first place.
• Paths: scripts/build/dockerfiles/build.Dockerfile, scripts/build/dockerfiles/entrypoint.sh, scripts/build/version.mk. BuildKit cache goes to bin/.buildkit (bin/ is already the gitignored tool directory).

5.2 The Dockerfile

Single stage named tools, so the default target and --target tools agree and no stage-naming mistake is possible. Base is debian with apt pinned to a snapshot date, because a base-image digest does not freeze the contents of the apt repository at the time of the issue’s reading of “pinned”:

# syntax=docker/dockerfile:1.7
# No version numbers anywhere below, and no defaults on any of these: the
# Dockerfile never decides a version. See §5.4.
ARG DEBIAN_IMAGE     # debian:bookworm-slim@sha256:... from scripts/build/version.mk
ARG DEBIAN_SNAPSHOT  # from scripts/build/version.mk
ARG GO_VERSION       # from go.mod
ARG NODE_VERSION     # from mcp/ and canopy/ package.json
ARG LICENSE_EYE_VERSION   # from scripts/build/version.mk
ARG SOURCE_DATE_EPOCH=1700000000

FROM ${DEBIAN_IMAGE} AS tools

# Real comments, on their own lines — a `#` after an ARG value is not a Dockerfile comment.
ENV DEBIAN_FRONTEND=noninteractive \
    SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH} \
    TZ=UTC LC_ALL=C LANG=C \
    PATH=/usr/local/go/bin:/usr/local/bin:/usr/bin:/bin \
    GOPATH=/go GOMODCACHE=/go/pkg/mod GOCACHE=/root/.cache/go-build \
    GOTOOLCHAIN=local \
    GOFLAGS=-buildvcs=false

# apt is pinned to a snapshot, and validity checks are disabled because a snapshot
# is by definition "expired". This is what makes `make`, `git` and `ca-certificates`
# reproducible rather than "whatever the mirror serves today".
RUN printf 'deb [check-valid-until=no] https://snapshot.debian.org/archive/debian/%s bookworm main\n' \
        "${DEBIAN_SNAPSHOT}" > /etc/apt/sources.list \
 && rm -f /etc/apt/sources.list.d/debian.sources \
 && apt-get -o Acquire::Check-Valid-Until=false update \
 && apt-get install -y --no-install-recommends make git ca-certificates jq xz-utils \
 && rm -rf /var/lib/apt/lists/*

# Go and Node are installed from their official tarballs. Copying /usr/local/go
# out of a golang image, or node's binary out of a node image, silently loses
# PATH, GOPATH and the runtime shared libraries the binary needs; installing the
# tarball is the boring, correct way.
#
# Each download is verified against the checksum the vendor publishes for that
# exact file, fetched over TLS at build time. Nothing about a toolchain bump is
# recorded in this repository, so a version bump touches go.mod or the two
# package.json files and nothing else. A version with no published checksum fails here rather than
# installing unverified bytes.
ARG GO_VERSION
RUN set -e; \
    go_file="go${GO_VERSION}.linux-amd64.tar.gz"; \
    curl -fsSLo /tmp/go.tgz "https://go.dev/dl/${go_file}"; \
    expected="$(curl -fsSL 'https://go.dev/dl/?mode=json&include=all' \
      | jq -r --arg f "$go_file" \
        '.[] | select(.version == "go'"${GO_VERSION}"'") | .files[] | select(.filename == $f) | .sha256')"; \
    test -n "$expected" && test "$expected" != "null"; \
    echo "${expected}  /tmp/go.tgz" | sha256sum -c -; \
    tar -C /usr/local -xzf /tmp/go.tgz; rm /tmp/go.tgz

ARG NODE_VERSION
RUN set -e; \
    node_file="node-v${NODE_VERSION}-linux-x64.tar.xz"; \
    curl -fsSLo /tmp/node.tar.xz "https://nodejs.org/dist/v${NODE_VERSION}/${node_file}"; \
    expected="$(curl -fsSL "https://nodejs.org/dist/v${NODE_VERSION}/SHASUMS256.txt" \
      | awk -v f="$node_file" '$2 == f { print $1 }')"; \
    test -n "$expected"; \
    echo "${expected}  /tmp/node.tar.xz" | sha256sum -c -; \
    mkdir -p /usr/local/lib/nodejs; \
    tar -C /usr/local/lib/nodejs -xJf /tmp/node.tar.xz --strip-components=1; \
    ln -s /usr/local/lib/nodejs/bin/node /usr/local/bin/node; \
    ln -s /usr/local/lib/nodejs/bin/npm  /usr/local/bin/npm; \
    ln -s /usr/local/lib/nodejs/bin/npx  /usr/local/bin/npx; \
    rm /tmp/node.tar.xz

ARG LICENSE_EYE_VERSION
RUN --mount=type=cache,target=/go/pkg/mod \
    GOBIN=/usr/local/bin go install github.com/apache/skywalking-eyes/cmd/license-eye@${LICENSE_EYE_VERSION}

# The entrypoint is normalized to LF and made executable during the build, so a CRLF
# shebang or a lost mode bit from the host checkout can never make the image unusable.
COPY scripts/build/dockerfiles/entrypoint.sh /usr/local/bin/entrypoint
RUN chmod 0755 /usr/local/bin/entrypoint \
 && sed -i 's/\r$//' /usr/local/bin/entrypoint

# Smoke-test the toolchain at build time, not at first use.
RUN go version && node --version && npm --version && license-eye --version

ENTRYPOINT ["/usr/local/bin/entrypoint"]

Two deliberate divergences from CI, both stated rather than glossed over:

• GOTOOLCHAIN=local. CI’s setup-build-env explicitly resets GOTOOLCHAIN=auto after setup-go. local is chosen so a go.mod bump newer than the image fails loudly instead of silently downloading a different toolchain. §5.4’s cross-check makes that failure legible.
• A single canonical platform, linux/amd64, set in the build and run invocations. Apple Silicon would otherwise resolve a different image manifest and potentially a different dependency platform. A multi-arch digest alone does not make dependency selection identical.

5.3 The entrypoint

$1 = the make target(s) to run, e.g. "license-dep"

1. Cross-check the image’s Go against go.mod and its Node against the mcp and canopy engines.node; exit non-zero with a clear message on mismatch (§5.4).
2. Confirm /work is populated and LICENSE_EYE=/usr/local/bin/license-eye is honoured.
3. Run make -C /work "$@" with LICENSE_EYE overridden (§6.2).
4. Copy the six output groups from /work to /out, creating empty files where a group is absent so that deletions are represented.
5. Print the manifest (§4.1) for /out on stdout, so the host can tee it without reimplementing the walk.

5.4 Version sourcing and the pin file

scripts/build/version.mk is the single place these are written down; scripts/build/dockerize.mk only reads and forwards them, with no literal of its own:

GO_VERSION   := $(shell sed -n 's/^go //p' $(mk_dir)go.mod)
NODE_VERSION := resolved from mcp and canopy engines.node
DISTRO       ?= bookworm
PLATFORM     ?= linux/amd64
SOURCE_DATE_EPOCH ?= 1700000000
LICENSE_EYE_VERSION := $(shell sed -n 's/^LICENSE_EYE_VERSION := //p' $(mk_dir)scripts/build/version.mk)

Tags are derived, never stored. A go.mod bump needs no Dockerfile edit and no lock-file edit: the Go and Node tarballs are verified against the checksum each vendor publishes for that exact file, so a version bump touches go.mod or the two package.json files and nothing else. Only the base-image digest and the apt snapshot are recorded rather than resolved, because neither can be derived from a URL the way a tarball checksum can.

This is a deliberate weakening of one property: a hash recorded in the repository would also catch a compromised origin, whereas fetching the expected hash at build time only catches corruption and truncation. It is the trust model the rest of this build already accepts — go install verifies license-eye through the module proxy, and license-eye then runs npm ci against the npm registry — and it is worth one honest sentence rather than a silent swap.

make bump-build-image re-resolves the base-image digest via docker buildx imagetools inspect, writing version.mk atomically (temp file + mv) so a failed lookup cannot leave it half-updated. It is only needed when moving the Debian base image or the apt snapshot — never for a toolchain bump. Wiring it into Dependabot is out of scope, because Dependabot’s docker ecosystem tracks the Dockerfile and this design deliberately keeps no image reference in it.

5.5 Node version consolidation

24.6.0 is currently a literal in four places: .github/actions/setup-build-env/action.yml, prepare.yml, canopy.yml, flaky-test.yml. Note that the last two use setup-node@v4 while the first uses v6, and that canopy/package.json and mcp/package.json carry their own engine constraints. Consolidating on node-version-file turned out not to require moving canopy.yml and flaky-test.yml off setup-node@v4: v4 accepts node-version-file as well. The four call sites were switched in place and the majors left alone, since changing them is unrelated churn. The engines reconciliation is a real obligation and is enforced by scripts/ci/check/node_version.sh (P3), which fails on an exact-pin mismatch, on a floor the pin does not satisfy, and — deliberately — on a range form it does not understand, because a check that quietly passes an unhandled range is worse than no check.

6. Make wrappers and workspace isolation

6.1 Why not a bind mount of the worktree

A bind mount of the live worktree is the obvious approach and it is wrong here, for four independent reasons:

• license-eye runs npm ci and npm prune --production inside the container against the mounted tree. On a Windows or macOS host that replaces the host’s node_modules with Linux packages and symlinks; on Linux it leaves root-owned files that obstruct subsequent native work. A package-lock.json backup protects the lockfile and nothing else.
• The host path is unquoted in the first draft, and drive letters, spaces, shell path conversion and daemon locality all break or silently misbehave.
• A linked Git worktree puts .git in a file pointing outside the mounted directory, so container-side git ls-files cannot resolve the tree at all. This is not hypothetical — it is the layout of this very checkout.
• make and the recipes assume a POSIX host toolchain, which a Windows container would not provide.

6.2 Tar in, tar out

The container never mounts the worktree. The host streams a copy in and takes the outputs back:

host  --(tar, excluding .git, bin, node_modules, dist)-->  container /work
host  <--(tar of the six output groups only)---------------  container /out

Consequences, all intended:

• The host’s bin/, dependency trees and lockfile bytes are never touched. package-lock.json is transported read-only, so the existing mktemp/EXIT-trap save-restore in mcp/Makefile and canopy/Makefile has nothing to protect against and can be simplified or left alone.
• Uncommitted source edits — including a dependency bump the contributor has not committed — are included, because the copy is a filesystem copy, not git archive. (git archive would have silently excluded exactly the inputs a contributor is trying to re-license.)
• Git is never invoked in the container, so the linked-worktree problem does not exist.
• Paths with spaces and drive letters are a tar quoting problem, not a Docker volume problem.
• The cost is a full source copy per run, which is acceptable for a license-dep-scale operation and is avoided entirely in CI by generating from a fresh checkout.

6.3 The wrappers, and the contract they enforce

make docker-run TARGET=<target> runs any target in the pinned environment under two rules that are checked, not merely documented:

1. Git metadata is opt-in, with GIT=1. By default the tree is streamed without .git, so check, check-format and pre-push (which ends in check) cannot run there. Unenforced they fail deep inside go mod tidy with a message about unpublished internal modules — nothing like the real cause. The wrapper refuses them up front and names the alternative.

   With GIT=1 a self-contained .git is assembled and streamed as well, and they run: make docker-run GIT=1 TARGET="generate pre-push" completes in ~14 minutes. The assembly is not a copy, because a linked worktree cannot be copied: .git is a 4 KB file pointing at <common>/worktrees/<name>, the objects live in <common>, and the index lives in the worktree’s own gitdir. So the common directory is streamed and this worktree’s HEAD and index are layered on top, with commondir dropped. Verified against a real linked worktree: the reconstructed repository produces the same status, diff and git add --renormalize result as the original.

   Two consequences, both intentional. GIT=1 runs these as checks, not fixes: check-format’s git add --renormalize . edits the container’s copy of the index, which is discarded on exit, so a contributor who wants the tree fixed must run it on the host. And GIT=1 stops excluding the generated license artifacts, because otherwise git status would report every one of them as deleted.

2. A target that compiles Go needs the generated protos first. api/proto/** is build output, not source: make generate produces it, and CI receives it as a prepare-job artifact, so a plain checkout does not contain it. Without it, go mod tidy cannot resolve the module’s own packages — the first thing to break, and a confusing one. Passing generate as the first target produces them in the same container run:

   make docker-run TARGET="generate lint vuln-check"
   

   Measured in the pinned environment: make generate tidy completes in ~2m40s, where tidy alone fails. The cost is that the image carries only the license toolchain, so buf, the protoc plugins, golangci-lint, revive, ginkgo and govulncheck are installed at run time over the network, at the versions pinned in version.mk — reproducible in version, but not hermetic, and the bulk of that 2m40s. Baking them in would fix both at the price of a much larger image; see §10.8.

# Resource defaults mirror the existing `make test-docker` target (2 CPUs, 4 GB),
# so the container path is no heavier than the repo's established local-test path.
RUN_CPUS        ?= 2
RUN_MEMORY      ?= 4g
RUN_MEMORY_SWAP ?= $(RUN_MEMORY)
RUN_SHM_SIZE    ?= 256m

DOCKER_RUN_FLAGS = --rm --platform $(PLATFORM) \
  --cpus=$(RUN_CPUS) --memory=$(RUN_MEMORY) --memory-swap=$(RUN_MEMORY_SWAP) \
  --shm-size=$(RUN_SHM_SIZE) \
  -e GOMAXPROCS=$(RUN_CPUS) \
  -e NODE_OPTIONS=--max-old-space-size=2048 \
  -e SOURCE_DATE_EPOCH=$(SOURCE_DATE_EPOCH) -e TZ=UTC -e LC_ALL=C \
  -e LICENSE_EYE=/usr/local/bin/license-eye

# TARGET must be a *target-specific variable*: `foo: bar ; TARGET=x` would be a
# shell recipe, not an assignment, and the container would never receive it.
docker-license-dep:   TARGET=license-dep
docker-license-check: TARGET=license-check

docker-license-dep docker-license-check: $(BUILD_IMAGE) | $(STAGE_DIR)
	@tar -C $(mk_dir) --exclude=.git --exclude=bin --exclude=node_modules \
	      --exclude=dist -cf - . | docker run -i $(DOCKER_RUN_FLAGS) \
	      -v $(STAGE_DIR):/out $(BUILD_IMAGE):$(BUILD_IMAGE_TAG)
	@tar -C $(STAGE_DIR) -cf - . | tar -C $(mk_dir) -xf -
	@$(MAKE) check-license-outputs

• LICENSE_EYE=/usr/local/bin/license-eye is passed as a command-line Make variable inside the container. It overrides scripts/build/license.mk and propagates to every recursive $(MAKE), so no recipe changes and no symlink is ever written into the host’s bin/.
• The build itself: docker buildx build --platform $(PLATFORM) --load -f scripts/build/dockerfiles/build.Dockerfile --build-arg GO_IMAGE=$(GO_IMAGE) --build-arg NODE_IMAGE=... --build-arg DEBIAN_IMAGE=$(DEBIAN_IMAGE) --build-arg GO_VERSION=$(GO_VERSION) --build-arg LICENSE_EYE_VERSION=$(LICENSE_EYE_VERSION) --cache-from type=local,src=bin/.buildkit --cache-to type=local,dest=bin/.buildkit,mode=max -t $(BUILD_IMAGE):$(BUILD_IMAGE_TAG) . Every --build-arg is explicit. A bare --build-arg X reads the shell environment, and a plain Make variable is not exported to the environment by default.
• Runtime caches (/go/pkg/mod, ~/.npm) are a separate concern from the image’s BuildKit cache mounts: --load does not carry build-time cache mounts into docker run. Add a runtime volume for them only after a cold-cache run is proven correct.
• The native path is untouched. make license-dep on a healthy host still works exactly as before; §7.1 runs both and compares them.

6.4 Resource limits

The defaults are 2 CPUs and 4 GB of memory, chosen to match the existing make test-docker target rather than to invent a second convention. Rationale for a limit at all: a container that sees the whole developer machine’s core count will size the Go build and npm’s heap from it, and the resulting peak memory is both unpleasant on a laptop and non-reproducible across machines.

The last two are the ones that matter for determinism rather than just comfort: a Go or Node process sizing itself from host resources can make timing host-dependent, and the entrypoint’s SOURCE_DATE_EPOCH guarantee only helps if nothing else varies. All six are overridable (make docker-license-dep RUN_CPUS=4 RUN_MEMORY=8g) for constrained machines, and the overrides are recorded in the job log so a non-default run is visible.

The image build is limited separately, because docker buildx limits do not come from docker run flags:

• With the default Docker driver, the build shares the daemon’s resources and cannot be capped per invocation. If a hard cap is required, use a container builder: docker buildx create --driver docker-container --driver-opt memory=4g,cpu-quota=200000 --name banyandb-build.
• go install of license-eye is the only memory-hungry build step; --build-arg overrides or a larger cpu-quota are the escape hatch if it OOMs under the cap.
• Cold builds are the worst case for the memory ceiling (the module cache is empty). A cold-cache run must therefore be part of the first CI validation (P4 in §9), otherwise the limit is only proven on warm caches.

7. CI wiring

7.1 The ubuntu check job

Both paths run, each verified by the same manifest, and the two manifests are compared to each other:

- name: License artifacts (native baseline)
  run: make license-dep
- name: Capture native manifest
  run: make license-manifest > native.manifest

- name: License artifacts (Docker, canonical)
  run: make docker-license-dep
- name: Capture canonical manifest
  run: make license-manifest > canonical.manifest

- name: Native and Docker output must be byte-identical
  run: diff -u native.manifest canonical.manifest

The container runs under the §6.4 limits in CI as well, deliberately: ubuntu-latest has more cores and memory than a developer’s laptop, and letting the container size itself from the runner would mean the canonical path is validated under a different resource regime than contributors use it under.

This is not redundant with §7.2, and it resolves a real failure mode: a broken image must not be able to hide a license change, and a license change must not be able to hide a broken image. check-license-outputs (§4.3) runs in check-req, so it also guards jobs that never generate.

ci.yml currently passes setup-docker: 'false' to setup-build-env; the check job needs either a Buildx-capable builder or a documented fallback to the default Docker driver, whose --cache-to type=local support depends on its image-store configuration. The cache key must include scripts/build/version.mk, go.mod, the two package.json files, the entrypoint and the platform.

7.2 Cross-host coverage: build system yes, generation no

Two revisions got this wrong in opposite directions, so the current position is worth stating precisely.

Generation on every host — not done, by decision. An earlier revision specified a test-license-determinism.yml matrix that regenerated the artifacts on macos-latest and windows-latest and compared manifests. It was dropped, because after §3 and §6.1 the pipeline has no host-specific behaviour left to test:

• The working tree is streamed into the container over a pipe, so there is no bind mount, no path translation, no daemon locality and no node_modules to be rewritten (§6.1).
• Line endings are neutralized at the checkout by eol=lf (§3), and at generation by license-normalize-eol (§4), so core.autocrlf is not a variable in the output.

What remains is Docker’s own container and image handling, which is Docker’s to get right and is covered by its own test suite. A matrix here would have been paying for Docker’s regressions rather than for ours, and it is not free: macos-latest and windows-latest provide a Docker client but no Linux engine, so the jobs would have needed Docker Desktop with a Linux VM — fragile, slow, and a recurring source of runner breakage.

The one host-specific defect this issue was actually about — CRLF reaching a committed artifact — is caught by check-license-outputs (§4.3), which runs on every host via make check-req and fails on any CR byte. That is a stronger guard than a matrix, because it also fires for a contributor who never pushes a branch.

Build system on every host — done. Dropping the generation matrix initially left nothing at all running on macOS or Windows, which is the wrong trade: CONTRIBUTING tells those contributors to run the same commands as everyone else, and that should be demonstrated rather than asserted. .github/workflows/test-build-system.yml runs on all three hosts and checks:

• the Node resolver and the license verifier, in their failing modes as well as their passing ones, so a green step cannot mean the check silently did nothing;
• that the pinned image builds on that host, with one cheap container run that also asserts the image carries the Go, Node and license-eye versions this repository declares;
• that the wrapper’s driver logic holds there — right platform, no version literals, clean tree.

It is gated from ci.yml’s result job, so a macOS or Windows failure blocks a PR. shell: bash throughout, because the verifier is a shell script and a Windows runner would otherwise use cmd.

What is still not covered: the generation itself on macOS and Windows. If a Docker transport problem exists there, this workflow is what will surface it, since the image build and the wrapper are exercised even though generation is not. The native make license-dep remains a fallback.

7.3 Windows-native note

A native Windows host is supported. The image is Linux, but Docker Desktop runs Linux containers directly on Windows, so the container is not the obstacle. What the wrapper actually needs from the host is small, and is checked rather than assumed:

--user $(id -u):$(id -g) exists to keep the staging directory owned by the invoking user on a POSIX host, where a root-owned artifact cannot be copied back without sudo and poisons the tree that npm ci runs in. Neither applies on Windows: NTFS/DrvFs has no uid ownership, so the copied-back files belong to the Windows user whatever uid wrote them. The flag is therefore applied only when a POSIX id is found, and its absence is not an error.

This was wrong in the first implementation, which rejected any host without id and pointed at WSL2. That conflated “the container needs Linux” with “the host tooling needs POSIX”. The native path remains a convenience and, with §3 in place, needs no per-OS core.autocrlf instruction; if someone does hit CRLF on a native run, the LF assertion in §4.3 names the file and the fix is §3.

The canonical path requires only Docker. The native path is a convenience; with §3 in place it no longer needs per-OS core.autocrlf instructions, which is what the issue’s acceptance criteria demand. If a contributor hits CRLF on a native run, the LF assertion in §4.3 prints the offending path and the fix is the one in §3, not a per-OS workaround.

8. Documentation

• CONTRIBUTING.md — the license section becomes “have Docker; run make docker-license-dep”. Native make license-dep stays documented as the fast path, without OS-specific configuration.
• README.md — one line in the build/development section pointing at the canonical command, as the issue explicitly requests.
• Makefile — a comment in the ##@ Build targets block naming make docker-license-dep, make bump-build-image and make check-license-outputs, and stating that Docker is canonical for anything that produces committed license artifacts.
• Image maintenance — the §5.4 procedure, in scripts/build/README.md.

9. Phasing

10. Risks and open questions

1. Archive reproducibility is untouched. SOURCE_DATE_EPOCH is set in the image, but scripts/release.sh still does not normalize archive member times, order, ownership or gzip metadata. Split into its own issue rather than implying it is covered here.
2. Silent partial resolution. license-eye logs npm ci failures instead of failing (§1.2). The manifest’s “against committed” mode is the mitigation, and it only works because the six groups are enumerated. If a new JS project is added and forgotten in §1.1, its licenses become unverified — a .licenserc.yaml-adjacent manifest, or a check that the enumerated groups cover every LICENSE/licenses directory in the tree, would close that hole.
3. Canonical platform. Pinning linux/amd64 makes arm64 contributors run an emulated amd64 image, which is slower but deterministic. The alternative — allowing the native platform — reopens manifest-divergence risk. This design takes determinism and accepts the cost; revisit if the cost becomes real.
4. Source copy per run. §6.2 copies the tree. For a repository this size that is seconds, but it is per invocation and should be measured before the targets are advertised as a fast dev loop.
5. Generation is not exercised on macOS or Windows. By decision (§7.2); the build system is checked there, generation is not. The compensating control is check-license-outputs, which runs on whatever host the contributor is on.
6. Base-image maintenance is manual. make bump-build-image is a documented procedure, not automation, and it now only covers the Debian digest and the apt snapshot — a Go or Node bump is a one-line edit to go.mod or to either package.json. Deliberate for this design; it is the obvious follow-up if the image outlives the issue.
7. The 2 CPU / 4 GB default is inherited, not measured. It matches test-docker and is deliberately conservative, but no one has profiled license-dep under it. If a cold-cache run OOMs or thrashes, the first move is to raise RUN_MEMORY, not to remove the limits — an uncapped container is what made this class of problem host-dependent in the first place.
8. The container is a check environment for the full pre-push, not a fix environment. With GIT=1 the whole pre-push chain runs there (§6.3), but it runs as a check: the renormalization check-format would stage is discarded with the container, and the seven tools it installs come from the network at run time. A contributor wanting a fixed tree still runs make pre-push on the host.
9. Baking the remaining tools is unpriced work. The ~2m40s figure in §6.3 is for the run-time install path. An image with buf, the protoc plugins, golangci-lint, revive, ginkgo and govulncheck baked in would be hermetic and substantially faster, at a materially larger size. Nobody has measured that size, nor the CI minutes it would save.

11. Implementation notes — what changed from the design

Reconciling the design against the code found five things worth recording. Two are bugs the design would have shipped, and one of them was caught only because the verifier was built first.

11.1 CRLF normalization belongs in the generator, not the container

The design put CRLF normalization in the entrypoint. That is wrong, and §7.1 caught it: if only the container normalizes, make license-dep and make docker-license-dep produce different bytes, so the native-vs-canonical comparison fails for reasons that have nothing to do with the host OS.

It now lives in the shared license-dep target (license_manifest.sh normalize, wired as the license-normalize-eol prerequisite). Both paths normalize; both produce the same bytes; a Windows contributor’s native run matches CI.

11.2 The verifier found a real, previously invisible defect

Four committed artifacts carried CR bytes, copied verbatim out of node_modules:

dist/licenses/ui-licenses/license-sass.txt
dist/licenses/ui-licenses/license-tslib.txt
mcp/licenses/license-json-schema-typed.txt
canopy/licenses/license-color-name.txt

The committed blobs were already LF; the CRLF was only in working trees, invisible to git diff and to check-format (which compares blobs). This is precisely the class of difference the issue is about, and it only became visible once a raw-byte LF assertion existed.

11.3 debian:bookworm-slim cannot fetch a snapshot over https

The slim base image ships no CA store, so it cannot complete a TLS handshake with snapshot.debian.org — it would need the very package that would give it one. The snapshot is therefore fetched over http. That is not a downgrade: apt verifies the Release and Packages signatures against the archive keyring in the digest-pinned base image, and package payloads are checksummed against those signed indexes. apt-get update fails outright on a mismatch. Go and Node tarballs are fetched over https and verified against pinned SHA-256 sums.

11.4 The container must run as the host uid

Otherwise the artifacts written into the mounted staging directory are root-owned and the copy-back needs sudo — which fails outright on a WSL DrvFs mount, and would leave root-owned files in the tree. --user $(id -u):$(id -g) fixes both. Three consequences followed:

• The image has no WORKDIR: one baked at build time is created as root and is unwritable by the invoking uid. The entrypoint creates its own scratch directory under /tmp.
• The image’s caches move to /tmp when non-root, since /go and /root are not writable.
• A host without id is rejected up front with a pointer at WSL, rather than failing later inside a tar.

11.5 COPY_EXCLUDES cannot exclude dist wholesale

canopy/Makefile reads dist/LICENSE.tpl as an input template. Excluding dist broke it. Only the two generated paths inside dist are excluded; --exclude globs the whole path, so dist/LICENSE does not match dist/LICENSE.tpl.

11.6 Verified

Nothing has to be added to .licenserc.yaml’s ignore list for this: an earlier revision carried a generated .node-version, which needed a header exemption because a comment would have made the file unparseable. Reading engines.node from canopy/package.json removes the file, and with it the exemption.