Trace Pipeline Merge Optimization Plan

Status

Proposed for review. Do not treat the targets in this document as frozen acceptance gates until they are approved.

Current Result (Baseline)

The controlled mature-merge comparison uses the same frozen production selection for the pipeline-disabled and native retain-all variants. Each variant ran in five fresh resource-limited Docker processes. All ten runs passed the selection, maturity, row-count, secondary-index, and logical-ledger correctness gates.

Each retain-all run made two plugin calls, evaluated and retained 33,353 complete trace IDs, dropped no traces, and preserved all 147,126 span rows. Compressed logical write amplification over the core and both secondary indexes remained identical at 1.0009 times.

Compared with the pipeline-disabled medians, the native retain-all path currently adds approximately:

121.9 MiB of allocated bytes, an increase of 56.2%;
1.07 million allocations, an increase of 38.0%;
91.5 MiB of peak Go heap, an increase of 83.4%; and
98.4 MiB of peak RSS, an increase of 72.8%.

The differential allocation profile points primarily to stageRawTrace, blockMetadata.copyFrom, traceEvaluationStager.stage, and assembleStagedEvaluationBatch. Profile values overlap and must not be added together.

The retain-all median wall and CPU times are lower than the disabled medians in the opening comparison, but this is not evidence of a speedup. The disabled wall-time series has 9.87% coefficient of variation and the CPU profile is dominated by variable filesystem syscalls. Physical process write-byte counters are diagnostic only because both variants produce byte-identical logical output while those counters differ substantially.

Invariants

Every optimization must preserve the following behavior:

The production picker selects the frozen input IDs in the same dispatch order.
All physical blocks for one trace ID are assembled into exactly one logical sampler decision.
Trace ordering in the merged output remains valid.
Timeout, panic, malformed-verdict, and plugin-error paths fail open.
The per-trace oversized bypass never evaluates a partial trace.
The fragment guard pins and validates conjunction parts before allowing a drop.
Core and secondary-index output remains logically correct.
The two-hour merge grace is unchanged.
The pipeline-disabled path does not perform plugin staging or projection work.
Existing controlled correctness gates are not weakened to obtain better performance numbers.

Phase Boundary Criteria

Every phase has four components:

Scope. What is in this phase and what is not.
Exit gate. An explicit checklist that must pass before the phase closes.
Hard invariant. What must NOT change as a result of this phase.
Dependencies. Which phases must close before this phase can begin.

A phase does not start until its dependencies are met. A phase does not close until its exit gate passes every item. Phases are ordered sequentially within the Core Track; the Parallel Workstreams (Phase 8 and Phase 9) run independently and may overlap with any core phase.

A phase is considered closed only after re-running the relevant slice of the per-phase validation checklist below. Closing a phase authorizes the next phase to begin; it does not authorize accepting the original optimization targets.

Core Track

Phase 1 — Measurement Foundation

Scope. Establish variance-controlled, comparable measurement for every subsequent optimization.

Entries from original plan. P0.

Hard invariants. Existing correctness gates are not weakened. Logical write-amplification (not /proc/self/io) is the gate. /proc/self/io counters remain diagnostic only.

Exit gate.

CPU-set pinning in place alongside GOMAXPROCS=4.
Pre-dispatch and post-introduction pprof bases captured independently.
Environment capture (image digest, CPU set, filesystem, kernel, Go version, binary checksum, plugin checksum, storage device, clone method) recorded per suite.
Disabled/enabled series stability documented across ≥5 alternating runs.
Wall-time and CPU deltas are not compared until the disabled/enabled series are sufficiently stable.

Dependencies. None.

Boundary rationale. No structural optimization can be evaluated without first proving that the test harness can detect deltas. This phase is the gate for evidence, not for performance. It measures the controlled mature merge and the serialized integration workload already frozen by the performance-test design; capacity sweep and ingestion throughput calibration are outside this plugin-overhead plan.

Phase 2 — Staging-Budget Correctness

Scope. Repair memory-safety and accounting correctness in the staging byte budget before any structural change.

Entries from original plan. P1.

Hard invariants. Aggregate staging budget and per-trace oversized budget remain logically separate. Oversized fail-open behavior preserved.

Exit gate.

Metadata and structural overhead (maps, slice capacity, trace-group descriptors, verdict state) counted in staged bytes.
Maximum trace-count limit added per batch.
Benchmark derives staging budget from real container memory limit, not an unlimited no-op protector.
Failing staging-budget tests cover boundary, oversize, and oversized-bypass interactions.

Dependencies. Phase 1.

Boundary rationale. Pooling and arena work in later phases would hide budget bugs and break the fail-open contract if the budget is wrong. Correctness must precede performance.

Implemented accounting contract. The byte budget includes retained payload capacity, raw and decoded metadata, map buckets, staged-slice backing capacity, per-trace evaluation descriptors, exact-drop map reserve, verdict state, and transient projected-column copies. The aggregate byte cap, per-trace oversized cap, and logical trace-count cap are evaluated independently. A batch is flushed only at a complete trace boundary; a single oversized trace remains fail-open and never reaches the sampler partially. The trace-count cap is 65,536 unless the byte budget implies a smaller bound. Controlled benchmark reports record the container memory limit and all three derived staging limits, and readiness rejects a report whose staging memory limit differs from the recorded cgroup limit.

Phase 3 — Retain-All Structural Optimization

Scope. Eliminate the dominant retain-all allocation sources via contiguous arenas, pooled vectors, and incremental trace grouping. This is the phase that targets the 50% allocation/bytes reduction goal.

Entries from original plan. P2, P3, P4.

Hard invariants. Deep-copy lifetime guarantee preserved. Reader may overwrite buffers; timed-out plugin may outlive the merge call. Invalid trace ordering rejected. Fragment-guard conjunction parts pinned and validated before any drop.

Exit gate.

One contiguous arena per staged raw block (or bounded staging chunk) holds spans, tag values, and tag metadata.
Pooled: blockMetadata, staged-block slices, trace-group descriptors, sdk.TraceBlock vectors, trace-ID vectors, fragment-guard ranges, decision masks — each with capacity limits and oversized discard.
Logical trace groups built incrementally with one descriptor per trace (first/last block indexes, accumulated min/max ts, staged-byte estimate, fragment-guard range).
Pooled objects fully reset before reuse; race tests pass.
Controlled re-run demonstrates ≥50% reduction in allocated bytes and allocation count, ≥30% reduction in peak heap and peak RSS.

Dependencies. Phase 1, Phase 2.

Boundary rationale. P2/P3/P4 attack the same hot allocation sites (stageRawTrace, blockMetadata.copyFrom, traceEvaluationStager.stage, assembleStagedEvaluationBatch). Splitting them across phases would force re-validation of shared lifecycles repeatedly. They close together.

Implementation and measurement note. Raw spans, tag values, and tag metadata now share one arena per staged raw block. Raw metadata descriptors use a contiguous bounded vector, and logical trace groups are updated while blocks are staged rather than rediscovered before every decision. Staged blocks, group descriptors, SDK trace blocks, trace IDs, fragment-guard ranges, and decision masks have bounded reuse. A decision batch that times out is deliberately not reset or returned to a pool because the timed-out plugin goroutine may still read it. Raw arena retention is bounded across the whole cache, and raw metadata caches are bounded by object count; a per-object limit alone retained almost an entire merge and increased heap residency. Arena storage uses BanyanDB’s pkg/bytes.Buffer; all staging pools are registered through pkg/pool, with Discard balancing internal-pool lifecycle accounting when a timeout or capacity bound makes an object unsafe or unsuitable for reuse.

The focused BenchmarkStageRawTrace result is approximately 5,584 B/op and 8 allocs/op for the arena path versus 23,856 B/op and 125 allocs/op for the former per-value-copy shape. The complete package race run passes. A five-pair alternating controlled run before bounding aggregate pool retention preserved every correctness ledger and reduced the retain-all allocation count, but exposed excess post-merge heap retention. After bounding the caches, a diagnostic controlled retain-all run used 319.3 MiB allocated, 3,277,411 allocations, 161.8 MiB peak heap, and 180.7 MiB peak RSS while preserving 147,126 rows and all ledgers. Against the opening retain-all medians, those single-run deltas are approximately -5.8%, -15.5%, -19.6%, and -22.6%, respectively; they are diagnostic rather than an acceptance series.

The remaining target is therefore open. The profiles show that the first fresh-process batch must still allocate the deep-copied payload, while the default 512 MiB staging budget lets this seed complete in one decision batch and gives the pools no within-run reuse. Phase 4’s batch-size sweep is required to determine whether bounded chunk reuse can meet the allocation and peak-memory targets. Until that sweep completes, Phase 3 is structurally complete but not closed as a performance gate.

Phase 4 — Batch Size Tuning

Scope. Empirically select the smallest staging batch size that meets memory and throughput goals without excess plugin-call overhead.

Entries from original plan. P5.

Hard invariants. Logical evaluated trace count, per-trace verdict, and merged output byte-identical across batch sizes. Both byte and trace-count limits enforced.

Exit gate.

16 / 32 / 64 / 128 / 256 / 512 MiB sweep completed against the unchanged frozen seed.
Each run reports peak heap, peak RSS, allocated bytes, allocation count, plugin calls, traces per call, CPU time, wall time, logical write amplification.
Selected batch size documented with both byte and trace-count limits; alternative sizes ruled out with evidence.

Dependencies. Phase 3.

Boundary rationale. Cannot tune a knob whose dependent structure is still changing. Phase 3 must be stable before doing a controlled sweep.

Phase 5 — Decision Path Overhead

Scope. Reduce per-batch decision overhead in the chain execution and fragment-guard paths.

Entries from original plan. P6, P7.

Hard invariants. Timeout, panic, malformed-verdict, per-link fail-open, timeout circuit breaker, and concurrent merge safety all preserved. Conjunction snapshot and coverage catalog pinned before plugin evaluation; lazy allocation must not weaken publication-time revalidation.

Exit gate.

Sampler-wrapper slice built once per batch (or per worker) rather than per batch.
Chain conjunction mask reused within bounded lifetime.
Single retain mask for single-sampler chains.
Reusable decision worker (or empirical evidence that the current goroutine/channel/timer pattern is cheaper) selected.
Fragment-guard ranges stored in compact integers; bounded guard-range vectors pooled; drop-specific probe state allocated lazily.
AlwaysKeep verification: with no fragment-guard confirmations, no throughput regression.

Dependencies. Phase 3.

Boundary rationale. Lower priority than raw staging because the opening controlled round only makes two plugin calls. Following Phase 3 keeps the structural changes from masking any decision-path gains.

Phase 6 — Drop-Path Optimization

Scope. Reduce overhead in the deterministic-drop path. Does not begin until retain-all overhead is reduced.

Entries from original plan. P9.

Hard invariants. Secondary-index lookup remains exact. No probabilistic structure can authorize a deletion. Core filtering and secondary-index pruning costs reported separately.

Exit gate.

Duplication between per-batch mature-drop set and merge-wide dropped-ID set removed.
Each dropped trace ID stored once.
Compact exact lookup representation for secondary-index pruning evaluated and selected.
Bounded drop-set storage reused between batches.
Guard confirmation objects constructed only when plugin proposes a drop.
Sweep at 1% / 35% / 99% deletion ratios completed; fixed cost and nonlinear behavior documented.

Dependencies. Phase 4, Phase 5.

Boundary rationale. Different workload characteristic from retain-all. The original plan explicitly defers this until retain-all overhead is controlled; honoring that deferral keeps the comparison meaningful.

Phase 7 — Projection Path Optimization

Scope. Optimize the tag/span projection path used by real SkyWalking policies. The AlwaysKeep benchmark does not exercise this path.

Entries from original plan. P8.

Hard invariants. SDK read-only input contract preserved. Timed-out plugin goroutine cannot observe recycled memory. Deep-copy lifetime protection retained unless an equally safe ownership mechanism replaces it.

Exit gate.

Copied tag values and span bodies packed into column arenas.
Aggregate block and column vectors reused.
Columns decoded only when requested by the combined plugin projection.
Ref-counted immutable batch buffers (if adopted) demonstrated safe with timed-out plugins.
Real-policy benchmark (e.g. error/status rules + regex/tag rules) shows measurable improvement.

Dependencies. Phase 3.

Boundary rationale. Fundamentally different code path from the retain-all work. Separate phase keeps the diff reviewable and prevents attribution confusion in the final integration report.

Parallel Workstreams

Phase 8 — Common Merge I/O Investigation

Scope. Investigate filesystem-level merge I/O (Fadvise frequency and granularity, small sequential writes, writer buffering and flush size, atomic-file synchronization, secondary-index writer behavior) as an ordinary merge investigation.

Entries from original plan. P10.

Hard invariants. Pipeline-disabled baseline must improve or be preserved. Any gain here is not credited as a plugin-framework optimization.

Exit gate.

Each axis profiled with before/after evidence.
Disabled-mode baseline regression test passing within tolerance.

Dependencies. None (parallel workstream).

Boundary rationale. Independent of the plugin framework. Runs as a workstream that may overlap with any core phase. The “not credited as plugin optimization” invariant prevents scope-creep across phases.

Phase 9 — Plugin-Local Benchmarks

Scope. Benchmark real plugin logic outside the merge framework so plugin-local costs are not confused with merge I/O or framework staging.

Entries from original plan. P11.

Hard invariants. Plugins are measured against representative complete-trace batches from the mature seed. Plugin-local fixes are kept separate from framework fixes.

Exit gate.

Independent measurements for projected tag/span decoding, complete-trace latency calculation, error and status rules, trace-ID hashing, regex/tag rules, verdict allocation, decision throughput across representative batch sizes.
Each plugin-local cost attributed to a specific function call site.

Dependencies. None (parallel workstream).

Boundary rationale. Outside the merge framework; can run any time after the mature seed is frozen. Parallel workstream.

Phase 10 — Final Integration

Scope. Run the serialized 24-hour SkyWalking workload with default settings and demonstrate that every acceptance gate is met.

Entries from original plan. P12.

Hard invariants. All gates from the original plan (selection, maturity, whole-trace, ledger) unchanged. Two-hour merge grace unchanged. No unfinished queued, running, or in-flight merge work.

Exit gate.

Nonzero plugin calls and evaluated traces.
Independently validated mature-trace deletion ratio near 35%.
Correct core and secondary-index ledgers.
Grace bypass for hot selections.
No OOM, timeout, panic, malformed verdict, or unexpected lossless retry.
No regression in pipeline-disabled baseline.
All proposed targets met (or explicitly accepted with quantified impact).

Dependencies. Phase 4, Phase 6, Phase 7 (and Phase 8 / Phase 9 if those axes feed into the final run).

Boundary rationale. Final acceptance gate. Only the targets frozen here become blocking.

Phase Dependency Graph

Phase 1: Measurement Foundation
        │
Phase 2: Staging-Budget Correctness
        │
Phase 3: Retain-All Structural Optimization
        │
   ┌────┴────┐
Phase 4   Phase 5
(Batch)   (Decision Path)
   │         │
   │         │
Phase 6   Phase 7
(Drop)    (Projection)
   │         │
   └────┬────┘
        │
Phase 10: Final Integration

(parallel at all times)
Phase 8: Common Merge I/O
Phase 9: Plugin-Local Benchmarks

Targets Becoming Blocking

These targets are proposals for review and are not yet frozen gates.

Metric	Current extra cost	Proposed first target
Allocated bytes	121.9 MiB	Reduce by at least 50%
Allocation count	1.07 million	Reduce by at least 50%
Peak Go heap	91.5 MiB	Reduce by at least 30%
Peak RSS	98.4 MiB	Reduce by at least 30%
Pipeline-disabled performance	Baseline	No regression greater than 5%
Logical correctness	Lossless	No change allowed

Until Phase 1 closes, no target is blocking. After Phase 4, the 50% allocation reduction and 30% heap/RSS reduction become blocking. After Phase 6, the deletion ratio near 35% becomes blocking. Wall-time and CPU targets remain non-blocking until CPU pinning and repeated runs reduce environmental variance below the current 9.87% CV.

Validation Checklists (per phase)

Each phase applies the relevant subset of the items below. Do not close a phase whose applicable items have not all passed.

Unit tests for buffer ownership, reset, pooling, trace grouping, ordering, and budget boundaries.
Timeout, panic, malformed-verdict, and plugin-error fail-open tests.
Oversized single-trace and multi-block trace tests.
Fragmented trace tests spanning selected and conjunction parts.
Raw and projected merge-path tests.
Race-enabled tests for shared pools, plugin instances, and decision workers.
The exact frozen controlled selection in both pipeline-disabled and native AlwaysKeep modes.
At least five fresh Docker processes per variant in alternating order.
Exact core and secondary-index ledger reconciliation.
Updated CPU, allocation, heap, block, and mutex profiles for one representative process per variant.

Do not proceed to a phase whose dependencies are still open. Do not proceed to deterministic dropping if AlwaysKeep output is not identical to the disabled output or if the current framework overhead has not been either reduced or explicitly accepted with quantified impact.

Mapping Back to Original P0–P12

Original	Phase	Reason
P0	Phase 1	Measurement foundation precedes everything.
P1	Phase 2	Memory-safety correction before any structural work.
P2	Phase 3	Combined with P3/P4; same hot allocation sites.
P3	Phase 3	Combined with P2/P4.
P4	Phase 3	Combined with P2/P3.
P5	Phase 4	Empirical tuning after structural changes.
P6	Phase 5	Combined with P7; lower priority than raw staging.
P7	Phase 5	Combined with P6.
P8	Phase 7	Different code path; separate phase.
P9	Phase 6	Deferred until retain-all is tamed; different workload.
P10	Phase 8	Independent workstream; not plugin-specific.
P11	Phase 9	Independent workstream; outside merge framework.
P12	Phase 10	Final integration.

Rejected Shortcuts

The following approaches are outside this plan:

special-casing AlwaysKeep so it bypasses the pipeline;
removing timeout-safe copies without a replacement lifetime mechanism;
writing retained output before Decide and attempting to roll it back;
making a two-pass merge the default without proving the I/O tradeoff;
reducing the two-hour merge grace to manufacture plugin calls;
relaxing the selection, maturity, whole-trace, or ledger correctness gates;
using probabilistic membership to authorize deletion; and
treating the current lower retain-all wall time as a framework speedup.