Skip to content

Comments

chore(agent-data-plane): Add OTTL filter processor unit-tests#1211

Merged
RiantZ merged 5 commits intomainfrom
azh/ottl_filter_tests
Feb 23, 2026
Merged

chore(agent-data-plane): Add OTTL filter processor unit-tests#1211
RiantZ merged 5 commits intomainfrom
azh/ottl_filter_tests

Conversation

@RiantZ
Copy link
Contributor

@RiantZ RiantZ commented Feb 23, 2026

Summary

Adds unit tests for the OTTL filter processor component (mod.rs)
All tests are in a #[cfg(test)] module and use ConfigurationLoader::for_tests and in-memory traces/spans.

Adds 18 unit tests for the OTTL filter processor in bin/agent-data-plane.
Tests are grouped as follows:

  • Configuration — absent ottl_config yields default (no drops); invalid config returns an error.
  • Build — invalid OTTL condition makes build fail; multiple valid conditions are parsed and combined with OR.
  • Filter logic — empty conditions keep all spans; condition true → drop, false → keep; resource.attributes and OR semantics; path returning non-bool errors and with error_mode: ignore span is kept.
  • Error modes — ignore / silent: eval error → span kept; propagate: eval error → span dropped.
  • transform_buffer — only matching spans are removed (remaining spans verified by label); when all match, trace has zero spans.
  • Performance tests (ignored by default, Criterion was not used because it required too many changes to the agent-data-plane project)

Run: cargo test -p agent-data-plane ottl_filter_processor

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

How did you test this PR?

Unit-tests

OTAGENT-868

@RiantZ RiantZ requested a review from tobz February 23, 2026 11:10
@RiantZ RiantZ requested a review from a team as a code owner February 23, 2026 11:10
@pr-commenter
Copy link

pr-commenter bot commented Feb 23, 2026

Binary Size Analysis (Agent Data Plane)

Target: b888e63 (baseline) vs 88b2f16 (comparison) diff
Analysis Type: Stripped binaries (debug symbols excluded)
Baseline Size: 27.39 MiB
Comparison Size: 27.39 MiB
Size Change: +0 B (+0.00%)
Pass/Fail Threshold: +5%
Result: PASSED ✅

Changes by Module

Module File Size Symbols

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [ = ]       0  [ = ]       0    TOTAL

@pr-commenter
Copy link

pr-commenter bot commented Feb 23, 2026

Regression Detector (Agent Data Plane)

Regression Detector Results

Run ID: 529fde20-969d-4a6f-819b-902f9836c3ad

Baseline: f348903
Comparison: 0805c9f
Diff

❌ Experiments with retried target crashes

This is a critical error. One or more replicates failed with a non-zero exit code. These replicates may have been retried. See Replicate Execution Details for more information.

  • otlp_ingest_metrics_5mb_cpu
  • otlp_ingest_logs_5mb_memory
  • otlp_ingest_logs_5mb_throughput

Optimization Goals: ✅ No significant changes detected

Experiments ignored for regressions

Regressions in experiments with settings containing erratic: true are ignored.

perf experiment goal Δ mean % Δ mean % CI trials links
otlp_ingest_logs_5mb_cpu % cpu utilization +2.85 [-2.37, +8.07] 1 (metrics) (profiles) (logs)
otlp_ingest_logs_5mb_memory memory utilization +1.66 [+1.13, +2.18] 1 (metrics) (profiles) (logs)
otlp_ingest_logs_5mb_throughput ingress throughput +0.03 [-0.10, +0.16] 1 (metrics) (profiles) (logs)

Fine details of change detection per experiment

perf experiment goal Δ mean % Δ mean % CI trials links
dsd_uds_512kb_3k_contexts_cpu % cpu utilization +8.32 [-48.72, +65.35] 1 (metrics) (profiles) (logs)
dsd_uds_1mb_3k_contexts_cpu % cpu utilization +6.05 [-47.62, +59.72] 1 (metrics) (profiles) (logs)
otlp_ingest_logs_5mb_cpu % cpu utilization +2.85 [-2.37, +8.07] 1 (metrics) (profiles) (logs)
dsd_uds_10mb_3k_contexts_cpu % cpu utilization +2.05 [-26.99, +31.10] 1 (metrics) (profiles) (logs)
otlp_ingest_logs_5mb_memory memory utilization +1.66 [+1.13, +2.18] 1 (metrics) (profiles) (logs)
quality_gates_rss_idle memory utilization +0.98 [+0.94, +1.02] 1 (metrics) (profiles) (logs)
otlp_ingest_traces_5mb_cpu % cpu utilization +0.78 [-1.37, +2.93] 1 (metrics) (profiles) (logs)
otlp_ingest_traces_5mb_memory memory utilization +0.52 [+0.27, +0.78] 1 (metrics) (profiles) (logs)
dsd_uds_100mb_3k_contexts_cpu % cpu utilization +0.37 [-5.05, +5.79] 1 (metrics) (profiles) (logs)
quality_gates_rss_dsd_ultraheavy memory utilization +0.32 [+0.19, +0.46] 1 (metrics) (profiles) (logs)
dsd_uds_500mb_3k_contexts_cpu % cpu utilization +0.29 [-1.05, +1.62] 1 (metrics) (profiles) (logs)
dsd_uds_10mb_3k_contexts_memory memory utilization +0.12 [-0.08, +0.31] 1 (metrics) (profiles) (logs)
dsd_uds_500mb_3k_contexts_memory memory utilization +0.08 [-0.11, +0.26] 1 (metrics) (profiles) (logs)
dsd_uds_500mb_3k_contexts_throughput ingress throughput +0.06 [-0.08, +0.20] 1 (metrics) (profiles) (logs)
otlp_ingest_logs_5mb_throughput ingress throughput +0.03 [-0.10, +0.16] 1 (metrics) (profiles) (logs)
dsd_uds_100mb_3k_contexts_memory memory utilization +0.01 [-0.18, +0.20] 1 (metrics) (profiles) (logs)
dsd_uds_1mb_3k_contexts_throughput ingress throughput -0.00 [-0.06, +0.06] 1 (metrics) (profiles) (logs)
otlp_ingest_traces_5mb_throughput ingress throughput -0.00 [-0.02, +0.02] 1 (metrics) (profiles) (logs)
dsd_uds_512kb_3k_contexts_throughput ingress throughput -0.00 [-0.06, +0.05] 1 (metrics) (profiles) (logs)
otlp_ingest_metrics_5mb_throughput ingress throughput -0.01 [-0.14, +0.11] 1 (metrics) (profiles) (logs)
dsd_uds_10mb_3k_contexts_throughput ingress throughput -0.01 [-0.16, +0.14] 1 (metrics) (profiles) (logs)
dsd_uds_100mb_3k_contexts_throughput ingress throughput -0.03 [-0.09, +0.03] 1 (metrics) (profiles) (logs)
quality_gates_rss_dsd_heavy memory utilization -0.04 [-0.17, +0.09] 1 (metrics) (profiles) (logs)
quality_gates_rss_dsd_medium memory utilization -0.14 [-0.33, +0.05] 1 (metrics) (profiles) (logs)
dsd_uds_512kb_3k_contexts_memory memory utilization -0.27 [-0.45, -0.09] 1 (metrics) (profiles) (logs)
quality_gates_rss_dsd_low memory utilization -0.29 [-0.45, -0.12] 1 (metrics) (profiles) (logs)
otlp_ingest_metrics_5mb_cpu % cpu utilization -0.55 [-6.58, +5.49] 1 (metrics) (profiles) (logs)
dsd_uds_1mb_3k_contexts_memory memory utilization -1.13 [-1.31, -0.95] 1 (metrics) (profiles) (logs)
otlp_ingest_metrics_5mb_memory memory utilization -2.32 [-2.54, -2.09] 1 (metrics) (profiles) (logs)

Bounds Checks: ✅ Passed

perf experiment bounds_check_name replicates_passed links
quality_gates_rss_dsd_heavy memory_usage 10/10 (metrics) (profiles) (logs)
quality_gates_rss_dsd_low memory_usage 10/10 (metrics) (profiles) (logs)
quality_gates_rss_dsd_medium memory_usage 10/10 (metrics) (profiles) (logs)
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 (metrics) (profiles) (logs)
quality_gates_rss_idle memory_usage 10/10 (metrics) (profiles) (logs)

Explanation

Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%

Performance changes are noted in the perf column of each table:

  • ✅ = significantly better comparison variant performance
  • ❌ = significantly worse comparison variant performance
  • ➖ = no significant change in performance

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

  1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

  2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

  3. Its configuration does not mark it "erratic".

Replicate Execution Details

We run multiple replicates for each experiment/variant. However, we allow replicates to be automatically retried if there are any failures, up to 8 times, at which point the replicate is marked dead and we are unable to run analysis for the entire experiment. We call each of these attempts at running replicates a replicate execution. This section lists all replicate executions that failed due to the target crashing or being oom killed.

Note: In the below tables we bucket failures by experiment, variant, and failure type. For each of these buckets we list out the replicate indexes that failed with an annotation signifying how many times said replicate failed with the given failure mode. In the below example the baseline variant of the experiment named experiment_with_failures had two replicates that failed by oom kills. Replicate 0, which failed 8 executions, and replicate 1 which failed 6 executions, all with the same failure mode.

Experiment Variant Replicates Failure Logs Debug Dashboard
experiment_with_failures baseline 0 (x8) 1 (x6) Oom killed Debug Dashboard

The debug dashboard links will take you to a debugging dashboard specifically designed to investigate replicate execution failures.

❌ Retried Normal Replicate Execution Failures (non-profiling)

Experiment Variant Replicates Failure Debug Dashboard
otlp_ingest_logs_5mb_memory baseline 8 Failed to shutdown when requested Debug Dashboard
otlp_ingest_logs_5mb_throughput baseline 4 Failed to shutdown when requested Debug Dashboard
otlp_ingest_metrics_5mb_cpu comparison 2 Failed to shutdown when requested Debug Dashboard

@RiantZ RiantZ added the type/chore Updates to dependencies or general "administrative" tasks necessary to maintain the codebase/repo. label Feb 23, 2026
@RiantZ RiantZ changed the title chore(agent-data-plane): Add OTTL filter processor unit-tests chore(agent-data-plane): Add OTTL filter processor unit-tests Feb 23, 2026
@RiantZ
Copy link
Contributor Author

RiantZ commented Feb 23, 2026

/merge

@gh-worker-devflow-routing-ef8351
Copy link

gh-worker-devflow-routing-ef8351 bot commented Feb 23, 2026

View all feedbacks in Devflow UI.

2026-02-23 15:59:07 UTC ℹ️ Start processing command /merge


2026-02-23 15:59:12 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 22m (p90).


2026-02-23 16:11:17 UTCMergeQueue: This merge request was updated

This PR is rejected because it was updated

@RiantZ RiantZ merged commit f058439 into main Feb 23, 2026
54 of 56 checks passed
@RiantZ RiantZ deleted the azh/ottl_filter_tests branch February 23, 2026 16:39
dd-octo-sts bot pushed a commit that referenced this pull request Feb 23, 2026
## Summary
Adds unit tests for the OTTL filter processor component (mod.rs)
All tests are in a #[cfg(test)] module and use
ConfigurationLoader::for_tests and in-memory traces/spans.

Adds 18 unit tests for the OTTL filter processor in
bin/agent-data-plane.
Tests are grouped as follows:
* Configuration — absent ottl_config yields default (no drops); invalid
config returns an error.
* Build — invalid OTTL condition makes build fail; multiple valid
conditions are parsed and combined with OR.
* Filter logic — empty conditions keep all spans; condition true → drop,
false → keep; resource.attributes and OR semantics; path returning
non-bool errors and with error_mode: ignore span is kept.
* Error modes — ignore / silent: eval error → span kept; propagate: eval
error → span dropped.
* transform_buffer — only matching spans are removed (remaining spans
verified by label); when all match, trace has zero spans.
* Performance tests (ignored by default, Criterion was not used because
it required too many changes to the agent-data-plane project)

Run: `cargo test -p agent-data-plane ottl_filter_processor`

## Change Type
- [ ] Bug fix
- [ ] New feature
- [x] Non-functional (chore, refactoring, docs)
- [ ] Performance

## How did you test this PR?
Unit-tests

[OTAGENT-868]

[OTAGENT-868]:
https://datadoghq.atlassian.net/browse/OTAGENT-868?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ f058439
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mergequeue-status: rejected type/chore Updates to dependencies or general "administrative" tasks necessary to maintain the codebase/repo.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants