Catalog B implementation tickets and verification contracts

Status: T02/T03/T05 are merged in PR #1417; T07/T08 are merged in PR #1424; T10/T12/T14 are supplied by open PR #1432; T15/T16/T17 and exact count-distinct composition are implemented in this PR 4 branch. Implementation does not imply that every historical performance gate passed: T02 remains inconclusive/stopped, and successful 100k/1m owning-query throughput is deferred. Current scope and validation are in the PR 4 verification record. The PR 1, PR 2, PR 3 and ticket records retain their original source bases and evidence.

The design-review base was 012115c5; PR #1412 merged as 15635181b. PR 4 integrates main at 86659118 with PR #1432. This document defines the language contract, batch engine design and compiled-plan reuse design. All listed capability commands are implemented; a recorded resource rejection is not successful throughput. Existing QLB-T01 labels and measurements retain their historical meaning.

Four-PR delivery

Each PR carries its own generated cases, focused benchmark evidence and language documentation. This split does not broaden individual benchmark workloads.

Parent QLB: deliver Catalog B through the existing Query RPC

Classification: tracking parent. Its 11 capability tickets are listed below. Each delivers an executable query capability or a substantial execution optimization, with real RPC end-to-end proof and a benchmark of that same behavior. The parent owns composition and release acceptance; drafting does not publish GitHub issues.

Ticket sizing rule: define the delivered query operation and the performance question before accepting a ticket boundary. Its benchmark must exercise the new execution path and measure latency, CPU, allocation or retained-state cost against an explicit baseline/reference. Adding a generic benchmark command to an error, syntax spelling or helper change does not make it a sufficiently sized ticket. Those changes and their tests belong inside the capability that needs them. Do not split error handling, numeric finalization, fixtures or instrumentation into independently completable tickets. Internal implementation steps may be small; the capability closes only after its full e2e and benchmark gates pass.

Fixed boundary: QL grammar/planning, liaison execution and Go-only extraction of existing native batch execution. Protobuf measure requests and QL share native scan/reduction operators; QL adds typed liaison operators above that seam. Reuse PR #1412’s RecordBatch, owners, schema source and admission machinery. No changes to protobuf/generated API contracts, RPCs, HTTP, persisted formats or native wire formats; no Rust runtime or Arrow IPC dependency. SQL operators stay on the liaison. Shared native Go buffering/ownership and read/dispatch compatibility repairs are included with their regression tests, as described in the PR 1 verification record.

Readiness: PRs #1412, #1417 and #1424 are merged. PR #1432 supplies the expression/filter dependency. This branch implements the final original Catalog B capabilities with their generated cases and scoped benchmarks. Fresh validation belongs to the PR 4 record; historical ticket evidence retains its original source revision and verdict. Completing this language subset does not provide relational joins, CTEs or full DPO application parity.

Dependency and benchmark map

IDs are preserved to keep existing code, artifacts and references unambiguous. Retired IDs are mapped below and must not be queued as separate tickets.

Suggested delivery order: T02, T03, T05, T07, T08, T10, T12, T14, T15, T16, T17. T07 and T10 can develop independently after T03. Native multi-aggregate execution follows a correct local reference; caching ships with invalidation, concurrency and resource bounds together. These are completion boundaries, not a requirement to write every internal component in one patch.

Verification required by every child

Real endpoints and independent fixtures

Extend the existing shared measure cases and register the new table in both test/integration/standalone/query/vectorized_test.go and test/integration/distributed/query/vectorized_test.go. The latter runs a liaison and two data nodes. Use the real BydbQLService.Query client, write fixtures through existing services, wait for visibility outside measurement, and assert actual measure_result values or gRPC errors. Never substitute a local transformer or fixture scan for these end-to-end checks.

The current test/cases/measure/data/data.go helper requires a native request YAML and only checks the QL result variant. It does not verify QL result values. The completed T01 work adds a separate QL-only verifier using .ql, #!params: and independently authored expected measure-response YAML/error specifications. Native equivalence is an additional oracle where applicable, not a prerequisite for expressing QL tests. Preserve timestamps when significant; ignore only explicitly nondeterministic identity/trace fields. Compare unordered rows as a multiset until ORDER BY applies. Check types, nulls, family/field names, hidden-slot exclusion and ordered rows.

Use this small fixture F for the examples below. time = T + offset, with T a fixed UTC millisecond aligned to the Unix-epoch 2-second grid and covered by the test retention window; host is an entity tag, zone a nullable string tag, v a nullable INT64 field and f a nullable DOUBLE field. rid is a unique integer tag for identifying rows, not an incidental storage SID. A separate array tag labels is specified in T14. Existing schemas have no NOT NULL flag: non-null grouping proofs must come from verified native guarantees or exact predicates such as host IN (‘a’,‘b’,‘c’), never just observing this fixture. Ensure distributed rows actually cover both nodes, and add a replica/version fixture to verify deduplication rather than assuming placement.

F has six rows, COUNT(v)=4, SUM(v)=12, AVG(v)=3.0, and AVG(f)=1.8. Host a has AVG(f)=0.5. Empty/null-only and >100-row/group fixtures are separate extensions, so expected values do not change silently. Floating results use an explicit tolerance (absolute 1e-12 for these small values); integer counts and identities compare exactly. Do not generate goldens with the new evaluator.

Commands and evidence

Each capability below names E(Tnn) and P(Tnn) for its own behavior. These are mandatory command recipes. Existing support work has labels measure-ql and QLB-T01; retain them as compatibility coverage. Its runnable E(T01) is:

go test ./test/integration/standalone/query -count=1 -timeout=30m \
  -run '^TestPropertyIntegrationQuery$' \
  -ginkgo.label-filter='measure-ql && QLB-T01' -ginkgo.fail-on-empty

go test ./test/integration/distributed/query -count=1 -timeout=30m \
  -run '^TestPropertyQuery$' \
  -ginkgo.label-filter='measure-ql && QLB-T01' -ginkgo.fail-on-empty

Replace QLB-T01 with the owning ticket ID for E(Tnn). Run all cases listed in that ticket in both topologies. Fail on an empty selection, skipped required cases or missing expected test names. Once T05 lands, each applicable feature case also runs cold, warm with changed parameters, and uncached with identical semantics. Add focused package tests first, using the ticket’s listed packages:

make test-ci PKG='./pkg/bydbql ./banyand/liaison/grpc'

make test-ci includes race detection. Substitute the relevant package list; pkg/query/ql is a proposed module, not present on the base. Exercise cancellation, owner poisoning and concurrent binding in focused tests. Cluster-wide failure tests still go through the real RPC; test-only clocks/observers/fault hooks may control scheduling or inspect counts, but cannot replace storage or QL execution. Every ticket records its RED command/result, GREEN counts, revision and benchmark artifact path before completion. Do not count a source-text assertion as activation.

Benchmark infrastructure and shared gates

Extend test/integration/distributed/querybench, including its config, workload, orchestrator and report handling with one scenario per active capability as it lands: qlb_t02, qlb_t03, qlb_t05, qlb_t07, qlb_t08, qlb_t10, qlb_t12, qlb_t14, qlb_t15, qlb_t16, qlb_t17. Keep implemented qlb_t01 as the unchanged-production baseline; retired IDs do not get new standalone scenarios. This harness starts real endpoints and is Docker-gated. Its entrypoint is TestDistributedQueryBench, not a Go Benchmark function. Keep native scan_all/top_with_filter as controls. The standalone benchmark suite currently skips its workloads and cannot satisfy these gates. One real distributed RPC benchmark per ticket is mandatory; both topologies must pass E(Tnn).

The existing P(T01) baseline command is:

DQB_ENGINE=measure DQB_MODES=vec \
DQB_SCENARIOS=qlb_t01 DQB_CARDINALITIES=1024,100000 \
DQB_QUERY_WORKERS=1 DQB_QUERY_ITERATIONS=1000 DQB_WARMUP_ITERATIONS=20 \
DQB_PROFILE=1 DQB_REPORT_DIR=/work/.omx/bench-reports/qlb-t01-r1 \
test/integration/distributed/querybench/run-docker.sh --cpus 4 --memory 8g

For P(Tnn), substitute its scenario and a unique report directory. Run five isolated repetitions, changing r1 to r2 etc.; retain all reports. Every scenario must emit its named variants. Timed variants require expected row count/checksum, timings and allocation records; zero samples, skipped scenarios or missing variants fail performance verification. Explicitly designated overload variants report admission outcomes and resource ownership without latency/throughput samples; their expected rejection does not fail the resource gate. Use 1,024 and 100,000 input rows unless a ticket specifies otherwise. Also run 1,000,000 rows for T07, T08, T12, T14, T15, T16 and T17. Bounded low-budget resource-stress variants may fail with the asserted error rather than return a result. Such a variant measures resource bounds of its owning query capability; it does not replace that ticket’s successful-query benchmark. Ordinary parse/bind errors stay in correctness tests. Keep setup and visibility waits outside timing. Validate results before timing and at sampled points during measurement so a fast wrong/error path cannot pass.

Every benchmark records p50/p95/p99 RPC latency, throughput, CPU profile, allocations/bytes per query, input/output rows, leaf calls, operator peak retained bytes, lease charges and post-close live reservations. Add planning/cache metrics when T05 lands. These process-wide profiles include the client, liaison and data nodes; use focused planner/kernel benchmarks for a planning CPU claim. Counters become required when their owning ticket lands; T01 marks unavailable operator/cache observations as unavailable, never as measured zeros.

These are proposed acceptance budgets, not measured claims:

• G1, compatibility: compare unchanged legacy QL and native controls with PR #1412 and the immediate prerequisite revision on the same machine/container, dataset, schema, concurrency and Go build. Median-of-five p50 and allocated bytes/query must be <=1.10x baseline; p95 <=1.15x. T01 captures the baseline. Save uncertainty and raw runs; an unstable comparison is inconclusive, not a pass. Freeze comparison inputs before optimizing. New unsupported SQL has no fictitious old-version latency baseline.
• G2, bounded resources: charge all live buffers/state, including native retained frames and final response, to the shared query budget. At 16 MiB and 256 MiB limits, either return the correct result or ResourceExhausted before an over-budget allocation. Live execution reservations and owner pins return to baseline after Close/cancellation. The current lease is monotonic: report its cumulative charges separately; do not assert that spent charges are refunded. Concurrent overload must reject explicitly with ResourceExhausted, preserve correct results for completed requests, and allow a subsequent query to succeed after all transports close. Unexpected errors, leaked ownership, process failure or failed recovery fail G2. Keep configured limits unchanged. T01 only captures existing resource observations; shared accounting and its enforcement start in T02 and remain mandatory for subsequent production changes.
• G3, per-ticket work: each leaf below adds a numerical work/retention or speed gate to G1/G2. Instrument operation counts in test builds. For new SQL, benchmark its real RPC and its specified reference/algorithm; do not claim lower latency merely because an internal counter improved.

Where a ticket names K(Tnn), add focused BenchmarkQLB/Tnn sub-benchmarks under pkg/query/ql, with independently written scalar/reference code in _test.go:

go test ./pkg/query/ql -run '^$' -bench '^BenchmarkQLB$/^T10$' \
  -benchmem -benchtime=2s -count=5

Replace T10 as required. Require matching benchmark records; Go succeeds on an empty regex match. Use the same inputs and output checks for both algorithms. These supplement P(Tnn), never replace it. Cache planning benchmarks live in banyand/liaison/grpc and are named explicitly in T05.

Capability tickets

T02 — Deliver shared native/QL batch execution

Supporting work already complete: former T01 supplies the QL-only verifier, independent F fixtures, four real RPC cases and the five-run qlb_t01 baseline. Its invalid-query case remains a correctness assertion. The production batch cutover and its measured execution results are recorded in T02 verification.

Performance question: what latency, allocation and retained-buffer cost does the shared batch boundary introduce or remove relative to the existing RPC path? Measure wide/hidden projections and aggregate outputs, not just helper call counts.

Boundary/activation: factor DistributedPlan output before iteratorFromBatches[WithSchema]/vectorizedMIterator; activate both dquery.measureQueryProcessor.Rev’s existing collector and QL’s NativeScan through that Go-only seam. Factor common group/revision/topology preparation so QL cannot bypass it. Preserve native request semantics, hidden columns, trace/status and final serialization. Inject shared accounting and explicit schema/Close ownership.

E2E cases: raw-and-aggregate-equivalence, hidden-columns-and-empty, multi-group-and-replica-dedup, and gate-error-cancel-budget compare existing goldens and statuses, including an early canceled client and a denied group. Assert both real callers activate the seam; QL creates zero intermediate DataPoints. Repeated-column views survive native descriptor reuse/poisoning.

Benchmark: P(T02) compares legacy raw and aggregate QL against T01 plus a wide hidden-column variant. G3: zero intermediate DataPoint construction, no live owner or reservation delta after 1,000 queries, and G1/G2. Document native eager frame residency rather than claiming this extraction makes execution streaming.

Checks/readiness: E(T02), P(T02); focused ./banyand/dquery, ./banyand/liaison/grpc, ./pkg/query/vectorized/measure/.... The review fixes pass 147 cases per topology, six protected/small-budget real RPC cases, focused unit/race checks and normal/tagged lint. V12’s full 20-pair small matrix passes 16 of 18 performance checks; QL raw and aggregate p50 remain inconclusive, so the large matrix is held and T02 is not accepted. A separate fixed CPU-placement diagnostic did not consistently reduce variation. A later direct v10/v12 comparison found no material latency difference; the user stopped the fresh acceptance rerun after ten complete pairs, with no final verdict. The v10 full pass is retained as a historical checkpoint. See T02 verification for the frozen sources, exact intervals, resource proofs and all retained outcomes.

T03 — Execute typed raw queries with aliases and time predicates

Performance question: how much does typed binding/projection cost, and how much native work does exact time pruning save compared with a complete residual scan?

Boundary/activation: bydbqlService.Query compiles immutable typed templates and creates fresh Scan/Projection/Filter/Limit state. Add source shorthand, raw AS, SlotID/type/nullability/source metadata and existing-response mapping. Legacy parameterized measure queries use the template/instance seam without changing native semantics. Unfinished aggregate shapes retain legacy lowering or are explicitly rejected until T07/T08 admits them.

Time binding supports fixed INTERVAL, one execution clock, WHERE/TIME intersection, contradiction-to-Empty and exact millisecond boundary arithmetic. Pure-time OR can prune by an envelope only while retaining its full residual. A residual prevents native aggregate/final-limit pushdown. T12 adds mixed field/time OR.

E2E cases: raw-aliases projects host AS node and v AS value with F’s types/nulls; repeated-source preserves independent descriptors for repeated projections; duplicate-and-ambiguous-name rejects invalid output names before scans. Cover both source spellings, multiple tag families, __ql collisions, non-measure dispatch, absent extended limit, LIMIT 0, legacy default limits and >100 extended raw rows. time-boundaries: time > T+1s AND time <= T+3s returns rid {3,4}; equality T+2s returns {3}. time-or returns {1,6}; time-intersection-empty performs no scan. fresh-now-and-fractional-boundary covers a changed injected clock/parameters, sub-millisecond bounds and checked domain extremes through the real RPC.

Benchmark: P(T03) compares raw aliases/plain and wide repeated projections, exact time predicates versus equivalent legacy TIME, and residual OR versus its complete-scan reference. Vary time selectivity and changed bound ranges while retaining text. Record RPC latency, allocations, copied bytes and scanned rows. G3: no per-row name/type resolution; metadata-only aliases copy zero value bytes; one clock capture per request; zero scan calls for Empty/LIMIT 0 and one logical native scan otherwise (one storage leaf call per data node). Exact time queries retain their native pruning, with G1/G2.

Checks/readiness: E(T03), P(T03); focused ./pkg/bydbql, ./pkg/query/ql, ./banyand/liaison/grpc. Builds on T02’s shared execution implementation. Alias mapping, time boundaries, ownership and final pagination are one raw-query contract; none is a separate ticket. The implementation, 142 generated cases per topology, five-run capability benchmark and resource proofs are recorded in T03 verification. An additional compatibility comparison was stopped with partial results and no gate verdict. This does not change T02’s previously recorded gate outcome.

T05 — Reuse compiled plans safely under concurrent OAP traffic

Performance question: do warm compiled templates reduce planning CPU/allocation, and does cold-miss coalescing prevent duplicate work without harming concurrency?

Boundary/activation: the Query caller adds the separate bounded compiled cache from plan reuse. Ship coherent schema/revision/watch-generation capture, complete dependency validation, registry-fallback bypass, count/byte caps, in-use handles and fresh binding/authorization/operators as one capability. Shared cold compilation follows remaining waiters, not the first caller’s context. Bound compile concurrency/in-flight keys; publish once per stable version, cancel all-waiter work and remove failed entries. Literal queries keep bypass behavior.

E2E cases: same-text-new-values returns different correct F rows on hits; clock-array-limit-null-rebind covers fresh now, empty IN, LIMIT 0 and typed null; schema-replace-delete-recreate, multi-group-dependencies and permissions-and-fallback cover full invalidation, unrelated updates, authorization and uncached fallback. Inject lower/same revision watch changes and compile races through test hooks with actual RPCs. cold-burst releases 32 callers for one key and compiles once while each returns its bound result. first-waiter-cancel, all-waiters-cancel, evict-in-flight and schema-change-during-compile establish lifetime safety under race/owner poisoning; failures are not cached.

Benchmark: per the implementation scope decision, P(T05) compares compiled caching enabled versus disabled for exactly two query shapes: simple legacy and a wide typed projection with eight parameterized time predicates. Keep 1,024 input rows, eight RPC workers, 20 checked warmups and 1,000 timed requests per variant; alternate two bindings without changing query text. Use five repetitions in 4-CPU, 8-GiB containers. Report RPC latency, process CPU/allocations and cache counters separately from the focused planning measurement. Cold bursts, invalidation, cancellation and count/byte bounds are correctness cases, not extra performance scenarios. The former working-set, churn, previous-ticket comparison and worker-count matrices are outside this ticket’s benchmark scope. Add BenchmarkQLBPlanReuse/T05 in banyand/liaison/grpc; run with -run '^$' -bench ‘^BenchmarkQLBPlanReuse$/^T05$’ -benchmem -benchtime=2s -count=5`.

G3: 1,000 warm requests perform zero analysis/optimization/recipe builds and record 1,000 hits; cache-disabled execution compiles every request. One compile per synchronized stable-key cold burst is checked by generated E2E cases. For a wide typed projection with eight parameterized time predicates, warm planning median ns/op and B/op each <=0.80x typed uncached. Compile and retained-plan caps include in-use retired plans; canceled work leaves zero in-flight entries. Preserve existing QL semantics and request resource ownership; do not rerun earlier tickets’ performance matrices. Report simple-query and RPC results separately; planning savings do not imply an end-to-end speedup.

Checks/readiness: E(T05), P(T05), focused planning benchmark; ./pkg/bydbql, ./banyand/liaison/grpc and the schema adapter’s owning package. Blocked by T03. Use established run.Go helpers. Invalidation, concurrency and bounds cannot be deferred to independent fixes after enabling reuse.

T07 — Execute complete SQL aggregation over raw batches

Performance question: how do raw aggregation throughput, allocations and retained group state scale with rows, groups and nullable inputs compared with an independent scalar reference?

Boundary/activation: the QL aggregate lowering chooses one complete native raw scan plus local HashAggregate for extended COUNT/SUM/MIN/MAX/AVG over named columns. AVG/MEAN uses float64 SUM/non-null COUNT finalization, sharing state with matching output calls; never native MEAN. Introduce the shared typed/null-aware group codec and dense group-ID state arrays through this real caller. Scalar empty COUNT is int64 zero, other aggregates null; grouped empty input has no groups. Native scan deduplication happens before local aggregation. The original T07 scope preserved count-distinct/stage restrictions. The DPO #81 follow-up explicitly adds local count-distinct composition while preserving native pushdown restrictions.

E2E cases: aggregate-F produces count(v)=4, sum(v)=12, min(v)=0, max(v)=6; nullable-keys keeps zone NULL and empty string separate; null-only-and-empty checks scalar/grouped results; complete-input uses 513 groups plus replicas and does not truncate at 100. Native-null-key and floating COUNT shapes must select raw local execution. Native count precision beyond 2^53 is separately checked at the adapter/state boundary with synthetic counts, not billions of fake e2e rows.

average-F expects AVG(v)=3.0 and AVG(f)=1.8; sub-one-average expects host a AVG(f)=0.5. Zero count produces null; unequal shard populations use combined sum/count, not an average of averages. Keep legacy native MEAN fixtures unchanged.

Benchmark: P(T07) uses 1/1,024/N groups, null mixes and batches spanning 65,536 rows; compare single/multiple aggregates and AVG versus SUM+COUNT+AVG with shared state. K(T07) compares dense states with an independent scalar reference. G3: one native raw leaf, exact N input rows after dedup, state cardinality equals actual groups, and fixed-width aggregate state grows with groups rather than rows. Report key/state capacity bytes separately from native residency; G1/G2 apply.

Checks/readiness: E(T07), P(T07), K(T07); focused ./pkg/query/ql, ./pkg/bydbql, ./banyand/liaison/grpc. Builds on T03. Bind/query tests expose new multi-output semantics; null-collapsing legacy helpers are not the oracle.

Implementation status: local aggregation and generated correctness coverage are implemented. Both resource-limited topologies pass all 215 selected specs, including 48 T07 specs. Five focused RPC repetitions pass at 1,024/32,768 rows; the local kernel also executes one million groups. The original successful 100,000/1,000,000-row RPC performance gate remains unmet: inherited native frame reservations reject these queries before the QL aggregate starts. These smaller measurements do not replace that requirement. The 16-MiB real-RPC resource probe also remains blocked during native readiness. See T07 verification for results, resource limits and retained evidence.

Scope decision: disk spilling is deferred to a later ticket, with its own generated E2E coverage and focused benchmarks. T07 retains current memory limits and ResourceExhausted behavior. The blocked measurements above remain recorded with their original outcomes.

T08 — Execute native multi-aggregate queries with keyed merge

Performance question: when does native aggregate fan-out/merge reduce cost versus T07’s one raw scan, and what are the cost/bounds of multiple branches and high-cardinality keys?

Boundary/activation: aggregate capability lowering chooses native SUM/MIN/MAX/ exact COUNT leaves where equivalence is proven, otherwise T07. Add auxiliary exact non-null COUNT state when a finalized aggregate loses empty/null information; deduplicate visible and auxiliary calls. Remove child final pagination, bound branch concurrency and merge batch columns by the full group tuple plus bucket timestamp. All leaves share the parent lease, bound values, deadline and authorization checks. Branch failures fail the query.

Lower eligible AVG/MEAN to SUM and exact non-null COUNT, finalize as float64 and reuse matching visible/auxiliary calls. Unsafe count precision, nullable keys or lost aggregate information select T07. AVG’s state/finalization is part of the aggregate operation, not an independent division helper ticket.

E2E cases: shared float SUM/COUNT/AVG/MEAN outputs, including repeated AVG, use two branches over exact non-null entity selections. Integer MIN/MAX/COUNT use three branches. All-null and missing groups prove count-based normalization to null, and two entity keys prove full-tuple merging. The original sum-max-repeated-call three-leaf example needs refinement: native integer SUM wraps and native float MAX has different nonfinite semantics. Such combinations remain valid SQL through T07; no small fixture proves native equivalence. bucket-and-multiple-keys additionally names bucket timestamps and deliberately permuted branch arrival. T08 tests the merge’s timestamp key at the operator seam; extended bucket grammar and its real-RPC coverage remain T16 work and are not claimed as passed by T08. missing-null-empty-groups validates union/normalization or the documented local fallback. branch-failure-cap-and-denial covers >100 groups, finite complete-scan-cap exhaustion via a test seam, failed child gates and cancellation before any unauthorized leaf dispatch.

native-local-average-parity reuses T07’s fractional/sub-one/empty and unequal partition goldens with identical SQL semantics. sum-count-average-reuse proves native branch reuse, including auxiliary counts for null normalization.

Benchmark: P(T08) compares native split/merge, forced-local same-semantics reference and equivalent individual native queries on a fixed dataset. The focused protocol measures float AVG, shared float SUM+COUNT+AVG, integer MIN/MAX/COUNT and an integer AVG fallback control. Integer SUM overflow is covered by E2E tests; no earlier ticket benchmark dimensions are added. G3: eligible AVG needs at most two distinct leaves, exposing its matching SUM/COUNT adds no leaves, and unsafe AVG makes one local raw scan. For other combinations, leaf count equals unique eligible visible/auxiliary calls (three for integer extrema above). Intermediate DataPoints are zero; merge work <= sum of input group rows plus one output visit per group; no branch receives a fresh full budget. K(T08) covers different branch order and high key cardinality; G1/G2 apply. Record split-versus-raw latency/cost without assuming pushdown wins.

Checks/readiness: E(T08), P(T08), K(T08); focused ./pkg/query/ql, ./pkg/bydbql, ./banyand/liaison/grpc. Builds on T07. Fixed-data oracles must not imply a common snapshot across separate native queries during concurrent writes.

Implementation: native lowering, full-tuple batch merging, SQL AVG/null finalization, safe whole-query fallback and shared branch ownership are implemented. The exact eligible subset and provider proofs are recorded in T08 verification. Both real topologies pass 246 selected specs, including 31 generated T08 specs with independent expected values. The focused K/P protocol compares eligible paths and the integer AVG fallback; it does not broaden earlier ticket benchmark matrices or change their outcomes. Five resource-limited repetitions at both input sizes pass correctness and work checks. At 32,768 rows, float pushdown is about 2.4x faster with about 79% fewer allocated bytes; at 1,024 rows, three-branch integer extrema is about 1.5x slower. The verification record retains this regression and the remaining bucket E2E boundary rather than treating safe native eligibility as a universal speedup. Native branches remain sequential, at most 32 distinct calls; query memory limits and monotonic admission charges remain enforced without spilling.

T10 — Evaluate typed vector expressions and masked conditionals

Performance question: does compiling scalar-or-column kernels avoid per-row interpreter cost, and does masked evaluation save work when branches are unused?

Boundary/activation: SELECT expressions activate the shared typed evaluator: arithmetic (+, -, , /, %), decimal/exponent floats, unary signs, CAST/::type, comparisons, AND/OR/NOT, IS NULL, IN, CASE and COALESCE. Preserve ::TAG/::FIELD and accepted legacy identifiers containing ‘-’/’’. T12 reuses these kernels in WHERE; HAVING follows in T14. Constants remain scalars; selections use one physical index space (nil=all, empty=none); split before uint16 capacity, skip zero-active batches and evaluate conditional arms only for eligible rows. Native batches already obey the 65,536 physical-row bound; reject oversized external input before uint16 indexing. No new parameter wire types.

Numeric decisions: INT/BIGINT=int64, FLOAT/DOUBLE=float64. Checked integer arithmetic; int/int division truncates toward zero, mixed numeric operations promote to double. Division by zero, overflow and invalid/out-of-range casts fail with InvalidArgument. Double-to-int truncates after finite/range checks. No implicit string conversion; CAST(? AS DOUBLE) accepts bound strings. Null propagates. Pin timestamp units/timezone and accepted cast pairs in binder tests.

Implemented timestamp units are UTC Unix nanoseconds in the signed int64 range. String casts use RFC3339Nano; timestamp-to-STRING normalizes to UTC. Boolean and untyped-null outputs require an explicit supported cast. Integer remainder of the minimum int64 by -1 is zero; division of that pair is an overflow error.

E2E cases: arithmetic-F: v+2={2,4,6,NULL,8,NULL}; cast-parameter changes bound string doubles; integer-division gives -5 / 2 = -2; numeric-errors covers invalid casts, overflow and zero division; lexer-compat preserves legacy names. Boolean output needs a supported cast; timestamps/arrays use existing response mapping. masked-case: CASE WHEN v=0 THEN 0 ELSE 8 / v END={0,4,2,NULL,1,NULL}; coalesce-F={0,2,4,-1,6,-1}; three-valued-cast distinguishes true/false/null; unreachable-invalid-arm never raises errors from masked rows. Cover typed null, empty/non-empty IN and changed parameters under repeated execution.

Benchmark: P(T10) measures five expression/control projections through RPC at 1,024 and 32,768 input/output rows over the existing nullable fixture, with a 256-MiB query budget. Every row is evaluated and returned. K(T10) compares independent scalar evaluation at batches 1/1,024/65,536 with dense/selected/null inputs. Vary constant/column/parameter expressions, null density, branch depth and first-arm hit rates 0/50/100%; measure ns/active-row, allocations and RPC cost. G3: constants allocate no full-length arrays; resolve offsets/types once per batch; 100% first-arm matches evaluate zero later-arm rows; median kernel ns/active-row <= scalar reference at batches >=1,024. G1/G2 apply. Error cases prove correctness within this capability; successful expression execution supplies its benchmark.

Checks/readiness: E(T10), P(T10), K(T10); focused ./pkg/bydbql, ./pkg/query/ql. Blocked by T03. Numeric policy, masking, casts and predicate kernels form one usable expression layer with a real projection caller.

Implementation: see T10 verification and the language guide. The evaluator uses existing native batches and scalar ConstColumn outputs. At the final projection boundary, constant arrays require tracked row headers for the unchanged serializer; their payload remains shared. The scalar-constant benchmark gate does not hide this array-output expansion.

T12 — Filter raw fields before grouping, projection and final limits

Performance question: how do residual filtering and downstream aggregation cost change with selectivity, hidden-column width and complete native input size?

Boundary/activation: capability classification keeps field predicates and inexact mixed predicates above the complete native raw scan. QL Filter narrows selection; local aggregates consume only surviving rows. Preserve required hidden inputs and forbid child aggregate/limit pushdown across any residual.

E2E cases: field-filter-F, v > CAST(? AS DOUBLE) with “1”, selects rid {2,3,5}; filter-before-aggregate returns SUM(v)=12 and COUNT(v)=3; float-filter f > 0.5 selects {2,3,5,6}. time-field-or proves an exact time envelope alone is insufficient. filter-before-limit-and-hidden-input uses >100 rows with matches after row 100, omits the filtered field from SELECT and still returns the correct final limit. FALSE and UNKNOWN both drop rows. Test a low-budget cancellation/error path.

Benchmark: P(T12) varies selectivity 0/1/50/100%, raw projection versus local aggregation and hidden-column widths. G3: exactly one logical raw scan (two physical data-node leaf RPCs in the distributed fixture), all required input rows examined (no intermediate 100/final-limit cap), zero native aggregate leaves when a residual remains, and aggregate input count equals filter TRUE count. K(T12) reports filtered rows/s and copied bytes; G1/G2 apply.

Implemented protocol: P(T12) has 16 variants at 1,024/16,384 input rows: 0/1/50/100% predicate thresholds × raw/scalar aggregate × one/two hidden predicate fields. Every query scans complete input, without LIMIT. The fixed schedule is 100 timed/5 warmups and 20 timed/3 warmups, one worker, 256-MiB query budget, five fresh 4-CPU/8-GiB Docker repetitions and CPU/heap profiles. Actual TRUE counts are recorded because partial 100-row cycles round the nominal percentages. K(T12), BenchmarkFilterT12, measures the complete public one-batch lifecycle at 1/1,024/65,536 rows, four thresholds and dense/selected-null inputs: 24 variants, five one-second samples. It reports examined and passing throughput, ns/examined row, output-copy bytes and charged selection/state bytes. It makes no speedup claim against unsupported old SQL. Output-copy bytes are structurally zero because the filter borrows columns; unit tests and K’s pre-timing identity checks prove that property. The RPC counter is a consistency invariant, not measured copy telemetry, and excludes charged boundary decoding scratch. Large-input 100k/1m resource probes use the same owning scalar aggregate query; native admission failures remain explicit blockers for successful large-input measurements. The 16-MiB readiness failure is recorded separately. The initial 32,768-row full-output smoke exceeds cumulative parent admission during final response encoding. This failure is retained; successful sizes are fixed before timing without changing native policy.

Checks/readiness: E(T12), P(T12), K(T12); focused ./pkg/query/ql, ./pkg/bydbql, ./banyand/liaison/grpc. Blocked by T07/T10. Float criteria never become unsupported tag-only protobuf criteria.

Implementation: see T12 verification and the language guide. The generated QL-only suite adds 48 specs with independent expected values and passes all 340 selected specs on both topologies, including unchanged legacy routing across groups with incompatible unused column types. Required T12 work counters, hidden columns, resource cleanup, bind reuse and bypass are checked. Safe native tag/time pruning never replaces a residual. T14 subsequently adds collection membership composition with new residual expressions; MATCH composition remains explicitly rejected.

T14 — Execute grouped queries with HAVING and membership predicates

Status: implemented. The generated suite adds 163 T14 specs and 35 coverage obligations; the full suite contains 503 selected specs per real Query RPC topology. The owning K(T14) and P(T14) protocols, results, resource probes and retained typed indexed-array membership restriction are recorded in T14 verification.

Performance question: what does filtering completed groups cost, and do hidden or repeated aggregates trigger unnecessary native work or result materialization?

Boundary/activation: post-aggregate QL Filter resolves grouped expressions and visible/hidden aggregate calls, evaluates three-valued truth and removes hidden slots at egress. HAVING precedes final pagination and follows complete merge. SELECT aliases are outside HAVING scope; reject nested aggregates/non-grouped sources. Include CONTAINS/NOT CONTAINS as aliases of legacy infix HAVING/NOT HAVING membership, preserving placeholder/list semantics. Clause and infix positions remain unambiguous. This spelling compatibility is a subtask, not its own ticket.

E2E cases: having-F, SUM(v)>3 by host, returns b/c with totals 4/6; hidden-aggregate selects only host and exposes no SUM field; having-null handles null-only groups. membership-and-having combines membership WHERE with post-aggregate HAVING and a T12 residual variant. Invalid references fail before dispatch. For labels [‘red’,‘blue’], [‘blue’], [] on three non-null rows, contains-single red returns the first row; not-contains-single returns the other two. Both spellings and scalar/list/array/null bindings follow independent legacy membership expectations.

Benchmark: P(T14) measures the full native/local aggregate -> HAVING pipeline at group selectivity 0/1/50/100%, hidden/repeated aggregate mixes, and membership array lengths 0/1/16/256. Compare against equivalent aggregation without HAVING and separately account for fewer output rows. K(T14) measures filtering group batches. G3: repeated SELECT/HAVING calls add zero branches; filter input equals completed groups, not raw rows; hidden slots are never serialized. Both membership spellings produce the same work; their alias alone is not the performance claim. G1/G2 apply. K(T14) isolates group-filter execution cost. P(T14) reports inclusive process CPU, allocated/reserved state, output rows/bytes and RPC latency; shared-kernel profiles cannot isolate HAVING CPU. Per-operator peak retained bytes and physical storage reads remain explicitly unavailable. Hidden-call variants omit the two visible AVG fields, so their latency comparison also includes narrower output. Actual full BydbQL response bytes are measured over the six independently checked timed samples after profiling stops. Contextual identifiers and indexed-array rejections have generated coverage. A live index-binding check precedes native dispatch even on compiled-plan cache hits; native indexed storage is unchanged.

Checks/readiness: E(T14), P(T14), K(T14); focused ./pkg/bydbql, ./pkg/query/ql. Prerequisites T08/T10/T12 are implemented. Grammar compatibility, reference errors and membership cases are acceptance details of executable grouped queries.

T15 — Implement DISTINCT on the complete visible tuple

Status: implemented. The generated suite adds 21 QL-only specs with changed bindings, plan reuse and cache bypass, including scalar/timestamp equality, cross-batch tuples, projection, aggregate/HAVING composition and pagination. The existing SQL group table runs with zero aggregate states under the shared query tracker. Named typed GROUP BY without aggregates supplies an equivalent reference; legacy native group-only routing is preserved. Arrays reject during binding, and general sorting remains T17. See T15 verification for real-RPC coverage, focused five-run benchmarks and the native materialization limits encountered by large resource probes. The memory follow-up adds compact pooled keys, unordered finite-LIMIT stopping and reduced native frame buffering, with focused before/after measurements. Explicit ordering and absent LIMIT retain complete deduplication; upstream aggregation and native materialization have separate bounds.

Performance question: how do distinct-tuple throughput and retained key bytes scale with duplicate ratio, cardinality and variable-width values?

Boundary/activation: DISTINCT lowers to T07’s group table with visible projection slots and no aggregate states. Hidden sort inputs cannot affect equality; reject ORDER BY expressions not derivable from visible DISTINCT output. Null equals null for grouping, distinct from zero/empty string; signed zero is normalized and all NaNs share one group key.

E2E cases: distinct-zone-F returns {x,y,NULL,empty string}; distinct-tuple uses two columns with deliberate concatenation collisions and replicated input; distinct-numeric covers null/zero/-0/NaN using independently specified writes; distinct-before-limit deduplicates >100 raw rows before final pagination. Timestamp/array tuples use explicit structural keys when their type is admitted; unsupported projected key types fail during binding.

Benchmark: P(T15) covers 1%, 50% and 100% distinct tuples, narrow/wide strings and empty input. G3: group count equals unique visible tuples, aggregate state count is zero, and retained key storage scales with unique tuple bytes rather than duplicate count. K(T15) measures hashing/equality throughput; G1/G2 apply.

Checks/readiness: E(T15), P(T15), K(T15); focused ./pkg/query/ql, ./pkg/bydbql. Prerequisites T07/T10 are implemented. The legacy first-row-per-group operator is not an acceptable replacement for tuple DISTINCT.

T16 — Evaluate computed group keys and resolve GROUP BY ordinals

Status: implemented. Multiple computed keys use typed batch evaluation before the shared SQL group table. Structural matching preserves parameter identity; ordinals resolve after star expansion. Bucket aliases use existing timestamp encoding, while local/native output and native merge identity retain the bucket. Mixed original-time/bucket keys remain independent and use local aggregation. The 65 generated QL-only specs pass cold/warm/bypass execution and native/local parity in both topologies, with 589 selected QL and 120 native compatibility specs per topology. Focused kernel and six-control RPC benchmarks compare equivalent stored/computed and explicit/ordinal grouping. See T16 verification for measurements, resource limits and remaining unavailable observations. Protobuf/native operators are unchanged; general sorting and spilling are outside this ticket.

Performance question: what is the cost of evaluating computed keys and feeding them to grouping, compared with equivalent precomputed/simple keys?

Boundary/activation: bind group ordinals after star expansion, validate them, evaluate each distinct key expression once per input batch and feed T07’s group table. Match SELECT/group expressions structurally. Expose TIME_BUCKET outputs through the existing native bucket semantics/response mapping; bucket time remains part of merge identity when native aggregation is eligible.

E2E cases: computed-group-F groups by CASE WHEN v >= 4 THEN ‘high’ ELSE ’low’ and returns sums high=10, low=2; ordinal-equivalence GROUP BY 1 gives the same result. ordinal-errors rejects zero, negative, out-of-range and aggregate targets; bucket-projection uses 2-second buckets with SUM(v) {2,4,6} at T,T+2s,T+4s. Test group-only queries, null keys and cache hits with changed expression params.

Benchmark: P(T16) compares equivalent explicit/ordinal grouping and simple/ computed keys across low/high cardinality. G3: explicit and ordinal forms have identical physical work counts; one evaluation per unique group expression per active row, one raw scan for computed grouping, and no per-row AST/name lookup. K(T16) measures key evaluation plus grouping; G1/G2 apply.

Checks/readiness: E(T16), P(T16), K(T16); focused ./pkg/bydbql, ./pkg/query/ql. Prerequisites T05/T08/T10 are implemented. Calendar functions/windows/gapfill are excluded.

T17 — Order and paginate results with full sort and compact top-k

Implementation status: implemented, with 48 new generated QL-only specs passing in both topologies. The complete measure selection runs all legacy regressions and retains nine known failures per topology. The planned 32-caller composition reaches the existing native receive-buffer pool limit in both sort modes; its benchmark result remains explicitly blocked. Focused measurements, source provenance and the exact acceptance limits are recorded in T17 verification. These results do not close the parent or the historical T02 performance acceptance.

Performance question: when does finite-limit top-k reduce CPU and retained bytes versus full sort, and do wide rows/large offsets keep state bounded?

Boundary/activation: one typed lexicographic comparator serves full sort and ORDER BY + finite LIMIT/OFFSET. Resolve keys once; preserve exact int64 comparison, bytewise strings, per-key ASC/DESC and NULLS FIRST/LAST. Defaults are ASC NULLS LAST and DESC NULLS FIRST; NaNs follow other non-null numerics in ascending order. Equal complete keys have unspecified order; add unique tie keys to test goldens. Checked uint64 k=OFFSET+LIMIT selects compact top-k; copy candidates into owned storage so a retained row does not pin its source batch. Final pagination follows all filter/group/distinct work. LIMIT 0 selects Empty. Both modes are budgeted; no spill or unbounded fallback on exhaustion.

E2E cases: sort-F, v DESC NULLS LAST, rid ASC, returns {5,3,2,1,4,6}; alias-and-hidden-sort checks alias precedence/output exclusion; group-sort returns c,b,a by SUM(v) DESC; exact-wide-values covers int64s above 2^53, strings/nulls/NaNs. distinct-order-scope applies T15 restrictions. topk-F adds LIMIT 2 OFFSET 1 and returns {3,2}; topk-full-sort-parity includes ties, nulls and hidden keys. limit-zero-and-overflow checks no scans/checked k; distinct-having-pagination prevents early pagination. wide-row-cancel-budget keeps sparse candidates from large batches and verifies eviction/Close/error cleanup. Test full-sort budget exhaustion too.

Benchmark: P(T17) measures full sort and top-k on 1/3/8 keys, sorted/reverse/ tied/random data, narrow/wide rows, k=10/100/1,000 and large offsets. Include 1 MiB source batches with one retained narrow candidate each. K(T17) compares independent tuple sorting with full sort, and top-k with full sort plus identical slicing. G3: sort expressions evaluate once per row; for N>=1,024 comparator calls stay <=4Nceil(log2(N)) on the full-sort workloads. Candidates <=k; candidate buffers <=2x live payload plus declared bounded descriptor/heap capacity; zero source-batch pins after copying and releasing current input. Separate native frame residency. For N>=100,000 and k<=100 on the same random data, top-k median ns/op <=0.80x full sort+slice. G1/G2 apply; report complete RPC and retained-memory costs as well.

Checks/readiness: E(T17), P(T17), K(T17); focused ./pkg/query/ql, ./pkg/bydbql, ./banyand/liaison/grpc. Prerequisites T05/T12/T14/T15/T16 are implemented, including composed cache/time/filter queries. Full sort can be an internal implementation step/reference, but the ordering ticket closes with both modes.

Parent completion and composed workloads

The parent closes only when every child has its E/P evidence and relevant K records, and the following composition cases run in both topologies. Attribute their ownership to the final implementing child (normally T17); they do not become an unbounded extra implementation ticket.

SELECT host AS node, SUM(v) AS total, MAX(v) AS peak
FROM metrics.qlb
WHERE time >= ? AND v > CAST(? AS DOUBLE)
GROUP BY host
HAVING SUM(v) > 3
ORDER BY total DESC NULLS LAST, node ASC
LIMIT 1 OFFSET 1

On F with start T and threshold “1”, this returns node b, total 4, peak 4. Change the same query’s threshold to “4”: HAVING leaves only c, OFFSET removes it, so the warm-cache result is empty. Change the start time to exclude c and test another empty result. Also compose CASE/COALESCE + ordinal grouping + AVG, and visible-tuple DISTINCT + multi-key top-k. Check cold/warm/uncached equivalence, fresh authorization, errors, traces and no cross-request mutable state.

Run the complete measure-ql label in both suites using the E commands without the ticket filter. Run existing measure/stream/trace/property/TopN compatibility cases affected by dispatch changes, and focused race/owner tests. Benchmark the composition as an additional qlb_t17 variant with changing OAP parameters, workers 1/8 for admitted-query timing under G1/G2; use 32 workers for the separate overload check described below. Report planned-versus-observed cache savings, native pushdown coverage, residual scan cost, and peak/live/cumulative memory separately.

Finally compare implementation changes against the PR #1412 base: protobuf and generated API files, native transport/frame encoders, persisted formats and data-node execution behavior must be unchanged. Go-only extraction of distributed batch results must retain existing native RPC golden results. Every feature has real endpoint proof and performance evidence; a green parser test or an empty benchmark selection cannot close a ticket.

Concurrent admission acceptance (revised 2026-10-18)

This user-approved revision separates performance from capacity protection for T17 and parent composition. It replaces the requirement that the 32-caller cohort must complete every query successfully under fixed limits. It does not change production admission policy or the earlier reports’ recorded verdicts.

Retain the four-CPU, 8-GiB container, no additional swap, existing protector policy and 256-MiB vectorized tracker setting. Keep native receive limits and fixture/query/binding sets fixed. Do not reduce concurrency, raise limits, retry failed calls into successes or discard failed runs to obtain a passing verdict.

For performance, any rejection invalidates the entire timed cohort. Do not compute a passing latency/throughput result from its successful subset. Preserve all repetitions and uncertainty. A resource-pass verdict cannot substitute for missing successful-query measurements at 100k/1m rows or compatibility evidence.

For overload, report attempted/successful/rejected counts, peak in-flight calls, error status and rejection boundary, correctness of every successful response, post-close live reservations/owners/pool references, and the recovery result. Do not assert that monotonic parent lease charges are refunded before the last transport owner closes. A rejection without verified cleanup and recovery fails the resource contract; unrelated ResourceExhausted errors must not be silently classified as the known native admission boundary.

A 32-client run can fully succeed through staggered completion. That is a valid concurrent correctness result, but does not prove overload rejection. Include a deterministic admission probe that holds receive reservations until capacity is exceeded, checks rejection without an allocation beyond the allowance, and checks cleanup/recovery. Do not require exactly 31 admissions in the lasting contract; that count describes the diagnosed configuration, not an API guarantee.

Report performance, overload and recovery verdicts separately. Rejected requests are never successful timing samples. Existing partial_known_32_blocked artifacts remain historical evidence from the former contract. The harness now implements revision 2026-10-18-v1, with four strictly successful timed variants and a separate validated overload/recovery artifact. It preserves historical report validation and rejects old blocked-only evidence for revised runs. Harness verification records the focused checks; this does not fill the separate large-input or full G1 performance evidence gaps.

PR #1412 compatibility measurement (2026-10-18)

The frozen compatibility comparison completed 80,000 timed RPCs over five paired repetitions of the T01 matrix. All median p50/p95/allocation estimates meet G1 budgets; six cases pass with uncertainty included and two 1,024-row QL p50 cases were inconclusive. A separate 20-pair confirmation passes both cases under the predefined one-sided uncertainty method, closing the compatibility follow-up for this matrix. The original verdict remains recorded. This is measurement evidence, not a change to thresholds or a restart of the historical T02 investigation. The retained baseline, current binaries and identical test-only harness are documented with full provenance.

Large-input acceptance (revised 2026-10-19)

Following the user-approved scope decision, successful throughput for T17’s 100k/1m owning queries is deferred until native input materialization can execute the complete workload within the fixed memory budget. This revises the T17 and parent completion requirements above: missing successful timing at those sizes is a documented deferral, not an outstanding Catalog B implementation task. No other child evidence requirement or performance threshold is waived. In particular, historical T02 inconclusive/stopped results remain unchanged; this revision does not establish that every historical performance gate passed.

The fixed 1m probe verifies the entire fixture, rejects all five owning queries with ResourceExhausted, and successfully executes all five recovery queries with zero recorded cleanup counters. At 100k, four owning queries reject and one composed binding succeeds exactly. That isolated success is feasibility evidence, not a successful timing cohort. Physical block layout can affect input admission, so rejection itself is not mandatory when an exact result fits.

Resume the deferred performance work only after a change makes the complete owning-query cohort admissible under the existing four-CPU, 8-GiB container and 256-MiB query tracker. Possible enabling work includes reducing native eager input residency or temporary-file spilling; selecting or implementing either is outside this revision. Then rerun fixture readiness, exact results, cleanup and recovery before measuring all required queries with fixed repetitions. A rejected call must never become a timing sample, and a successful subset cannot pass the cohort. Raising limits, shrinking the workload or rerunning an unchanged rejected cohort does not fulfill the deferred requirement.

No benchmark code, memory protection, SQL behavior, protobuf or persisted format changes accompany this acceptance revision. Spilling and retained-result session pagination remain explicitly deferred. There is no new Catalog B feature ticket.

Count-distinct composition follow-up

The existing native count-distinct operation now composes through the extended QL path with multiple aggregates, aliases, multi-key grouping, HAVING, general ORDER BY and final LIMIT/OFFSET. This addresses four single-measure composition gaps for queries expressed using the extended path: multiple aggregates, deterministic tie-breaking, complete aggregation without an implicit 100-group cap, and pagination after ordering. Use an alias or the extended source form to select extended scalar/grouped semantics; use ORDER BY count_alias DESC, unique_group_key ASC for ranked pages. Legacy TOP/native query behavior and defaults are preserved.

Distinct counts use complete raw input and retain exact (group, value) keys in the shared bounded QL grouping storage. Repeated distinct calls share state; ordinary COUNT/SUM state remains separate. The extended plan deliberately does not sum native distinct partial counts, which would be invalid when shard value sets overlap. Exhausted memory fails the query without partial results. Protobuf, native wire behavior and persisted formats are unchanged.

This is not full DPO #81 coverage. Its linked PostgreSQL SQL in issue #79 remains outside this follow-up: joins/CTEs, computed aggregate arguments, aggregate FILTER, additional string functions and exact decimal arithmetic are not added. Missing telemetry dimensions or identities lost during rollup require ingestion changes. Separate page calls still do not share a snapshot; spilling and session pagination remain deferred.

The scalar empty-result lifecycle correction was already delivered in PR #1417; this PR preserves it while extending local distinct composition.

Verification uses generated QL-only endpoint cases for scalar/grouped count composition, NULL and empty input, changed bindings, HAVING, tied ordering and pagination past 1,100 groups in both topologies. Operator tests exercise scalar types, duplicate values across batches, borrowed-buffer ownership, cancellation, memory rejection and zero retained reservations. A narrow operator benchmark compares ordinary COUNT and exact COUNT(DISTINCT) on identical inputs at two cardinalities; it measures added state/cost, not equivalent-result speedup.

Follow-up verification (2026-10-19): the full planner and QL operator suites pass, including querymetrics builds; focused distinct ownership/race tests pass. The full generator suite and the final affected artifact/oracle checks pass. Both standalone and distributed QL-generated selections pass 6/6 specs (five new composition scenarios plus the existing count-distinct scenario). Scoped lint reports zero issues. The deep-page fixture has 1,153 groups, offset 1,100 and seven returned rows. It projects rid, COUNT(DISTINCT rid) and SUM(v); multi-key grouping is tested separately with the smaller fixture. A wider three-tag control hit the host’s native 8-MiB query allowance even with ordinary COUNT; the final test preserves group count/page depth without raising budgets.

Empty scalar follow-up: legacy QL COUNT(DISTINCT) now completes a successful empty native result with one integer-zero row, before trace assembly and response memory accounting. Grouped results and pages beyond the scalar remain empty; explicit LIMIT 0 is retained in QL plan metadata. Native protobuf query behavior and nonempty native execution remain unchanged. Failure/cancellation responses cannot become zero counts, and the shared empty response is not mutated. The neutral generated QL-parent-child cohort covers nine scenarios, including idle filtered/global counts, grouped empty input, and scalar limit/offset bindings. Both topologies pass all nine scenarios in four-CPU, 8-GiB containers with no extra swap or query-budget overrides. Full liaison/planner unit suites, focused final regressions, generator artifact checks and scoped lint pass. Reports: /mnt/d/qlb-empty-scalar-standalone-report.json and /mnt/d/qlb-empty-scalar-distributed-report.json.

Reproduction commands, using Go 1.26.9:

make generate-ql-test-cases
go test ./pkg/bydbql ./pkg/query/ql ./test/cases/measure/qlgen -count=1
go test -tags=querymetrics ./pkg/bydbql ./pkg/query/ql -count=1
go test -race ./pkg/query/ql -run CountDistinct -count=1
go test -tags=querymetrics -count=1 -timeout=15m ./test/integration/standalone/query -ginkgo.label-filter=QL-generated -ginkgo.focus=count_distinct
go test -tags=querymetrics -count=1 -timeout=15m ./test/integration/distributed/query -ginkgo.label-filter=QL-generated -ginkgo.focus=count_distinct

Final endpoint reports are /mnt/d/qlb-count-distinct-final-standalone-report.json and /mnt/d/qlb-count-distinct-final-distributed-report.json; corresponding logs and the ordinary-count capacity control are retained alongside them. No protobuf changes or external issue updates were made.

The narrow operator benchmark completed five repetitions per case in a four-CPU, 8-GiB Docker container with no additional swap, GOMAXPROCS=4 and Go 1.26.9. Each operation consumes 65,536 integer rows. Median results:

This measures exact-set overhead, not a speedup comparison between equivalent queries or an RPC latency gate. Distinct state grows with unique values; repeated input values do not retain additional keys. Each operation checks its independent expected count and verifies zero tracked memory after Close. Retained capacity is the final set reservation, not peak process RSS. The benchmark excludes native input scanning and transport. Runner, raw five-run samples, binary digest, observed cgroup limits and summary are under .omx/dpo81-count-distinct/.