T01 verification and baseline

Historical ticket-stage record. Results, source revisions and acceptance outcomes below retain their original context. See the PR 1 verification record for the latest isolated branch checks and refreshed benchmark observations.

Revision: 012115c56652691d4163f69d0460dfa9b064191f plus the local T01 test changes, verified on 2026-10-15 using Go 1.26.9. Production execution, protobuf source, generated API contracts, native transport and persisted formats are unchanged. The engine capabilities remain planned and unimplemented. The revised ticket plan folds this completed verification work into T02’s shared batch-execution capability. This record preserves the original T01 labels, scenario, measurements and scope; it does not mark T02 complete.

Verification

T01’s RED condition was the absent QL-only result-verification and qlb_t01 scenario contracts on the base: the existing QL helper checked only the response variant. This is infrastructure coverage evidence, not a production defect claim. The first focused build also exposed missing ignored generated protobuf files and mocks. They were generated with existing buf plugins and a Go 1.26-built mockgen; no generated source is part of the requested change.

The following focused checks passed:

go test ./test/cases/measure/data -run 'Test(CompareQL|VerifyQL)' -count=1
go test -race ./test/cases/measure/... -count=1
go test ./test/integration/distributed/querybench -race -count=1
# Repository-pinned v2.14.0, built with Go 1.26.9:
golangci-lint run --new-from-rev=HEAD \
  ./test/cases/measure/... ./test/integration/distributed/querybench/...
bash -n test/integration/distributed/querybench/orchestrate.sh
git diff --check

The verifier rejects wrong values, types, ordering, timestamps, names, nulls and duplicate multiplicities, and rejects a wrong response variant or gRPC status. Only SID, version and trace are ignored; timestamps remain significant. Floating values have absolute tolerance 1e-12; integers and protobuf value types are exact. Review regressions also cover near-zero floats in both comparison modes and overlapping tolerance matches under every two-row permutation. These failed before the fixes. Float comparison now preserves the protobuf default zero, and unordered comparison finds a complete one-to-one matching rather than committing to the first approximate match. The focused tests and measure race suite passed again after these changes.

Both real endpoint suites passed all four T01 cases: legacy-select-golden, legacy-parameter-rebind, invalid-ql-status and native-equivalence.

TMPDIR=/mnt/d/qlb-t01-tmp go test ./test/integration/standalone/query \
  -v -count=1 -timeout=30m -run '^TestPropertyIntegrationQuery$' \
  -ginkgo.label-filter='measure-ql && QLB-T01' -ginkgo.fail-on-empty
TMPDIR=/mnt/d/qlb-t01-tmp go test ./test/integration/distributed/query \
  -v -count=1 -timeout=30m -run '^TestPropertyQuery$' \
  -ginkgo.label-filter='measure-ql && QLB-T01' -ginkgo.fail-on-empty

Standalone selected 4 of 943 specs: 4 passed, 0 failed (14.863 seconds). Distributed selected 4 of 626 specs: 4 passed, 0 failed (28.557 seconds). The other specs were excluded by the requested label filter. Raw logs are local at .omx/bench-reports/qlb-t01-e2e/{standalone,distributed}.log.

The initial standalone attempt failed before executing specs because the root filesystem was 97–98% full and storage rejected writes with STATUS_DISK_FULL. Using the workspace-backed temporary directory above resolved that environment failure without changing production configuration. The Docker orchestrator also keeps temporary files and its compiled binary on mounted report storage.

Fixture F is minute-relative within retention and aligned to the epoch 2-second grid. It uses the existing two-shard, two-replica group; the distributed suite has two data nodes. The six-row golden additionally verifies an obsolete row version is replaced and replicas do not multiply output. Before the liaison assertions, the distributed suite queries each data node directly through the existing cluster RPC. Each node must independently return all six golden rows with version 2, proving actual placement and replacement of the obsolete version.

After the review fixes, standalone passed 4/4 cases again (14.806 seconds), and distributed passed 4/4 (29.838 seconds). The distributed log records six version-2 rows from each of two distinct node addresses, followed by 28 vectorized dispatches and 10 frames encoded/decoded. Updated raw logs:

• .omx/bench-reports/qlb-t01-e2e/standalone-review-fixes.log;
• .omx/bench-reports/qlb-t01-e2e/distributed-node-observation.log.

The fixes affect verification only; the benchmark workload and production path are unchanged, so the five retained benchmark repetitions below remain the baseline. The querybench race suite also passed again. Full lint of the measure cases, both query integration packages and querybench reported zero issues after reordering the new report fields to satisfy the repository’s alignment check.

Docker baseline

All five prescribed Docker repetitions passed both cardinalities and report merge. The final reports contain 40 verified variant records and 40,000 timed real RPCs, with 80 nonempty CPU/heap profile artifacts. The prescribed command is executed sequentially for r1 through r5:

DQB_ENGINE=measure DQB_MODES=vec \
DQB_SCENARIOS=qlb_t01 DQB_CARDINALITIES=1024,100000 \
DQB_QUERY_WORKERS=1 DQB_QUERY_ITERATIONS=1000 DQB_WARMUP_ITERATIONS=20 \
DQB_PROFILE=1 DQB_REPORT_DIR=/work/.omx/bench-reports/qlb-t01-r1 \
test/integration/distributed/querybench/run-docker.sh --cpus 4 --memory 8g

Each repetition starts isolated scenario/cardinality processes. Within each process, native, native_rebound, legacy_ql and legacy_ql_rebound execute sequentially on one cluster with the same query projection and two tag bindings. Later variants can observe prior cache/heap residency. Each emits exactly 1,000 timed requests, latency percentiles, throughput, process-wide CPU/allocation observations and CPU/heap profiles. Setup, visibility waits, expected-response hashing and sampled full-result verification are outside measurement. The first, every hundredth and last response are retained for exact validation after profiling stops; retained sample memory remains part of the process-wide heap observation.

The output is 64 rows per binding at input cardinality 1,024 and 6,300 rows at 100,000. Expected checksums include timestamps, value types, names and values, ignoring only SID/version. Unavailable leaf-call, operator-retention, lease, post-close reservation and planning/cache counters are explicitly marked unavailable rather than recorded as zero. No G2 enforcement claim is made by T01.

Local raw evidence is retained at:

• .omx/bench-reports/qlb-t01-r{1..5}/distributed-querybench.{json,md};
• each repetition’s shards/ and profiles/ subdirectories;
• .omx/bench-reports/qlb-t01-logs/r{1..5}.log.

Docker-created artifacts are normalized to the host user’s ownership after runs. This captures the unchanged-production baseline for later tickets; it does not claim an optimization speedup or a comparative G1 pass.

Median-of-five and full run ranges (not confidence intervals):

The ranges expose run-to-run variation, especially the first larger-cardinality run. These observations freeze a baseline; they do not establish stable relative performance between native and QL or against a prior revision. Future G1 comparisons must use matching machine/container, dataset, schema, concurrency and Go build.

Local ignored evidence paths, relative to the repository root (not shipped as repository links): .omx/bench-reports/qlb-t01-r1/distributed-querybench.json through .omx/bench-reports/qlb-t01-r5/distributed-querybench.json, plus .omx/bench-reports/qlb-t01-logs/summary.md. Per-run Markdown reports contain percentiles, QPS, allocations and profile paths. This document preserves the reproducible commands and measured summary in the repository.

Environment: Linux amd64, Go 1.26.9, Docker image banyandb-distributed-querybench:go1.26, cgroup v2, 4-CPU quota and 8-GiB memory limit on a host reporting 32 logical CPUs. No competing test/build workload was run during timing. Existing periodic schema-registry activity remains included in the process-wide observations.