Skip to content

chore(ci): add final_status property on junit XML [APMSP-2610]#1681

Open
cbeauchesne wants to merge 2 commits intomainfrom
cbeauchesne/final_status
Open

chore(ci): add final_status property on junit XML [APMSP-2610]#1681
cbeauchesne wants to merge 2 commits intomainfrom
cbeauchesne/final_status

Conversation

@cbeauchesne
Copy link
Contributor

@cbeauchesne cbeauchesne commented Mar 6, 2026

What does this PR do?

Add a <property name="dd_tags[test.final_status]" value="pass" /> on every test case reported through junit

Motivation

Some repositories intentionally use mechanisms that ignore test failures (there are valid reasons to still execute those tests behind the scenes).

In these cases, the reported test status cannot be relied upon by Test Optimization to monitor repository health or trigger team notifications.

To ensure a consistent and unified approach, we will instead use final_status — the property specifically designed and used by the Test Optimization integration for this purpose.

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

@pr-commenter
Copy link

pr-commenter bot commented Mar 6, 2026 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2026-03-06 16:39:34

Comparing candidate commit 7b48a91 in PR branch cbeauchesne/final_status with baseline commit 0f3d46b in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 56 metrics, 2 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:single_flag_killswitch/rules-based

  • 🟥 execution_time [+8.921ns; +9.661ns] or [+4.694%; +5.084%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 160.467µs 161.071µs ± 0.212µs 161.038µs ± 0.104µs 161.163µs 161.477µs 161.596µs 162.404µs 0.85% 1.639 7.471 0.13% 0.015µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [161.042µs; 161.101µs] or [-0.018%; +0.018%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 240.351ns 251.439ns ± 12.908ns 246.444ns ± 3.417ns 251.228ns 286.060ns 290.613ns 291.236ns 18.18% 1.883 2.465 5.12% 0.913ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [249.650ns; 253.228ns] or [-0.711%; +0.711%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.303µs 2.375µs ± 0.022µs 2.373µs ± 0.005µs 2.382µs 2.420µs 2.428µs 2.433µs 2.55% -0.598 2.815 0.93% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.372µs; 2.378µs] or [-0.129%; +0.129%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 534.288µs 535.650µs ± 1.082µs 535.286µs ± 0.369µs 535.760µs 537.641µs 539.534µs 540.744µs 1.02% 1.933 4.418 0.20% 0.077µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1849302.715op/s 1866897.459op/s ± 3757.063op/s 1868160.917op/s ± 1289.283op/s 1869301.774op/s 1870378.470op/s 1870910.421op/s 1871648.098op/s 0.19% -1.917 4.316 0.20% 265.664op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 380.775µs 381.567µs ± 0.293µs 381.548µs ± 0.206µs 381.755µs 382.032µs 382.259µs 382.341µs 0.21% 0.155 -0.175 0.08% 0.021µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2615468.083op/s 2620769.930op/s ± 2013.108op/s 2620899.233op/s ± 1415.318op/s 2622276.093op/s 2623996.214op/s 2625376.513op/s 2626219.836op/s 0.20% -0.151 -0.175 0.08% 142.348op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 189.626µs 190.179µs ± 0.193µs 190.156µs ± 0.125µs 190.285µs 190.505µs 190.674µs 190.883µs 0.38% 0.478 0.717 0.10% 0.014µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5238819.654op/s 5258212.763op/s ± 5331.962op/s 5258850.393op/s ± 3449.162op/s 5262058.792op/s 5265495.284op/s 5268502.003op/s 5273526.316op/s 0.28% -0.470 0.707 0.10% 377.027op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 37.391µs 37.534µs ± 0.060µs 37.535µs ± 0.037µs 37.568µs 37.639µs 37.662µs 37.733µs 0.53% 0.233 0.292 0.16% 0.004µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26501850.397op/s 26642272.440op/s ± 42240.223op/s 26641805.213op/s ± 25981.077op/s 26669703.034op/s 26710690.124op/s 26729596.060op/s 26744233.434op/s 0.38% -0.222 0.280 0.16% 2986.835op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 45.776µs 45.918µs ± 0.058µs 45.913µs ± 0.034µs 45.950µs 46.019µs 46.076µs 46.170µs 0.56% 0.764 1.517 0.13% 0.004µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 21659205.872op/s 21777801.862op/s ± 27470.706op/s 21780531.862op/s ± 16323.384op/s 21796392.031op/s 21815224.734op/s 21826155.620op/s 21845577.496op/s 0.30% -0.753 1.484 0.13% 1942.472op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [535.500µs; 535.800µs] or [-0.028%; +0.028%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1866376.766op/s; 1867418.151op/s] or [-0.028%; +0.028%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [381.527µs; 381.608µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2620490.933op/s; 2621048.928op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [190.152µs; 190.206µs] or [-0.014%; +0.014%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5257473.805op/s; 5258951.722op/s] or [-0.014%; +0.014%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [37.526µs; 37.543µs] or [-0.022%; +0.022%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [26636418.351op/s; 26648126.529op/s] or [-0.022%; +0.022%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [45.910µs; 45.926µs] or [-0.017%; +0.017%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [21773994.686op/s; 21781609.037op/s] or [-0.017%; +0.017%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 4.947µs 5.035µs ± 0.057µs 5.037µs ± 0.042µs 5.067µs 5.149µs 5.157µs 5.162µs 2.49% 0.566 -0.488 1.14% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.027µs; 5.043µs] or [-0.158%; +0.158%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sdk_test_data/rules-based execution_time 145.344µs 147.443µs ± 2.004µs 147.101µs ± 0.392µs 147.547µs 149.170µs 154.846µs 168.411µs 14.49% 6.924 62.760 1.36% 0.142µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sdk_test_data/rules-based execution_time [147.165µs; 147.721µs] or [-0.188%; +0.188%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
single_flag_killswitch/rules-based execution_time 196.954ns 199.325ns ± 1.811ns 199.310ns ± 1.299ns 200.414ns 202.114ns 206.087ns 207.662ns 4.19% 1.154 2.716 0.91% 0.128ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
single_flag_killswitch/rules-based execution_time [199.074ns; 199.576ns] or [-0.126%; +0.126%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 10.607ms 10.639ms ± 0.015ms 10.639ms ± 0.008ms 10.647ms 10.660ms 10.679ms 10.718ms 0.75% 1.230 5.151 0.14% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [10.637ms; 10.641ms] or [-0.019%; +0.019%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.893µs 3.912µs ± 0.003µs 3.912µs ± 0.001µs 3.914µs 3.917µs 3.920µs 3.920µs 0.20% -1.585 15.471 0.07% 0.000µs 1 200
credit_card/is_card_number/ throughput 255096414.169op/s 255591260.110op/s ± 167953.305op/s 255604728.688op/s ± 76657.007op/s 255676965.159op/s 255820545.460op/s 255882244.038op/s 256859304.110op/s 0.49% 1.615 15.700 0.07% 11876.092op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 78.966µs 79.547µs ± 0.348µs 79.550µs ± 0.273µs 79.740µs 80.182µs 80.386µs 80.759µs 1.52% 0.591 -0.013 0.44% 0.025µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 12382458.773op/s 12571393.546op/s ± 54883.432op/s 12570722.162op/s ± 43065.597op/s 12619160.011op/s 12643600.074op/s 12661707.554op/s 12663726.404op/s 0.74% -0.570 -0.059 0.44% 3880.845op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 72.343µs 73.045µs ± 0.395µs 72.982µs ± 0.286µs 73.343µs 73.770µs 74.027µs 74.083µs 1.51% 0.503 -0.468 0.54% 0.028µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 13498402.705op/s 13690674.944op/s ± 73923.707op/s 13702012.842op/s ± 53845.807op/s 13751464.583op/s 13791162.826op/s 13821518.013op/s 13823129.526op/s 0.88% -0.483 -0.495 0.54% 5227.195op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.896µs 3.913µs ± 0.003µs 3.913µs ± 0.002µs 3.915µs 3.917µs 3.919µs 3.921µs 0.21% -1.044 8.474 0.07% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 255045177.934op/s 255560296.307op/s ± 171792.129op/s 255577823.449op/s ± 120662.506op/s 255674453.084op/s 255770120.105op/s 255816307.663op/s 256702136.103op/s 0.44% 1.063 8.606 0.07% 12147.538op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 69.156µs 69.653µs ± 0.336µs 69.578µs ± 0.226µs 69.868µs 70.264µs 70.596µs 71.000µs 2.04% 1.018 1.213 0.48% 0.024µs 1 200
credit_card/is_card_number/378282246310005 throughput 14084523.573op/s 14357153.804op/s ± 68931.528op/s 14372404.285op/s ± 46754.959op/s 14413201.071op/s 14436355.870op/s 14454060.731op/s 14460126.653op/s 0.61% -0.987 1.092 0.48% 4874.195op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 52.154µs 52.222µs ± 0.035µs 52.219µs ± 0.021µs 52.239µs 52.293µs 52.306µs 52.321µs 0.20% 0.650 -0.046 0.07% 0.003µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 19112908.211op/s 19148887.822op/s ± 12960.799op/s 19150230.466op/s ± 7842.632op/s 19158374.122op/s 19167009.647op/s 19169562.879op/s 19173804.748op/s 0.12% -0.647 -0.051 0.07% 916.467op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.028µs 6.037µs ± 0.008µs 6.036µs ± 0.002µs 6.039µs 6.043µs 6.074µs 6.115µs 1.30% 5.622 42.086 0.14% 0.001µs 1 200
credit_card/is_card_number/x371413321323331 throughput 163542966.399op/s 165641966.296op/s ± 226398.629op/s 165676175.768op/s ± 66304.030op/s 165740850.977op/s 165816877.796op/s 165870965.361op/s 165886572.628op/s 0.13% -5.571 41.400 0.14% 16008.801op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.895µs 3.913µs ± 0.003µs 3.913µs ± 0.001µs 3.914µs 3.916µs 3.919µs 3.921µs 0.20% -1.544 12.363 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 255062939.475op/s 255573395.557op/s ± 164615.501op/s 255580448.096op/s ± 91177.118op/s 255663276.661op/s 255789454.297op/s 255880314.591op/s 256761615.223op/s 0.46% 1.568 12.544 0.06% 11640.074op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 64.181µs 64.491µs ± 0.141µs 64.478µs ± 0.092µs 64.584µs 64.756µs 64.842µs 64.989µs 0.79% 0.541 0.324 0.22% 0.010µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15387176.879op/s 15506173.403op/s ± 33797.664op/s 15509084.202op/s ± 22163.362op/s 15528898.615op/s 15556388.604op/s 15567987.289op/s 15580942.949op/s 0.46% -0.527 0.299 0.22% 2389.856op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 57.836µs 58.097µs ± 0.159µs 58.066µs ± 0.100µs 58.184µs 58.414µs 58.515µs 58.598µs 0.92% 0.889 0.263 0.27% 0.011µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 17065408.653op/s 17212844.645op/s ± 47125.244op/s 17221776.967op/s ± 29711.801op/s 17249356.321op/s 17270138.826op/s 17283089.560op/s 17290132.521op/s 0.40% -0.877 0.235 0.27% 3332.258op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.893µs 3.912µs ± 0.003µs 3.912µs ± 0.001µs 3.914µs 3.917µs 3.918µs 3.919µs 0.17% -1.593 13.962 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 255171132.311op/s 255597758.024op/s ± 173141.847op/s 255606959.072op/s ± 93756.892op/s 255696931.649op/s 255830844.686op/s 255871390.119op/s 256882429.824op/s 0.50% 1.620 14.182 0.07% 12242.977op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 54.581µs 54.997µs ± 0.290µs 54.942µs ± 0.212µs 55.192µs 55.526µs 55.770µs 55.808µs 1.58% 0.708 -0.207 0.53% 0.020µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 17918458.020op/s 18183425.246op/s ± 95396.291op/s 18200900.959op/s ± 70404.646op/s 18260873.149op/s 18305695.348op/s 18316252.498op/s 18321500.865op/s 0.66% -0.688 -0.246 0.52% 6745.536op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 52.151µs 52.226µs ± 0.035µs 52.220µs ± 0.021µs 52.245µs 52.298µs 52.318µs 52.321µs 0.19% 0.609 0.046 0.07% 0.002µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 19112727.464op/s 19147673.920op/s ± 12708.654op/s 19149765.378op/s ± 7736.578op/s 19156461.195op/s 19166515.235op/s 19168613.016op/s 19175126.027op/s 0.13% -0.606 0.042 0.07% 898.638op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.029µs 6.040µs ± 0.013µs 6.036µs ± 0.002µs 6.039µs 6.072µs 6.108µs 6.111µs 1.24% 3.191 10.974 0.22% 0.001µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 163627519.343op/s 165559247.348op/s ± 365077.935op/s 165662479.604op/s ± 67797.797op/s 165723049.364op/s 165803354.500op/s 165852427.569op/s 165872844.241op/s 0.13% -3.173 10.824 0.22% 25814.908op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.912µs; 3.913µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/ throughput [255567983.397op/s; 255614536.823op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [79.499µs; 79.595µs] or [-0.061%; +0.061%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12563787.230op/s; 12578999.862op/s] or [-0.061%; +0.061%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [72.990µs; 73.099µs] or [-0.075%; +0.075%] None None None
credit_card/is_card_number/ 378282246310005 throughput [13680429.829op/s; 13700920.059op/s] or [-0.075%; +0.075%] None None None
credit_card/is_card_number/37828224631 execution_time [3.913µs; 3.913µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/37828224631 throughput [255536487.570op/s; 255584105.044op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/378282246310005 execution_time [69.607µs; 69.700µs] or [-0.067%; +0.067%] None None None
credit_card/is_card_number/378282246310005 throughput [14347600.557op/s; 14366707.051op/s] or [-0.067%; +0.067%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [52.217µs; 52.227µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [19147091.580op/s; 19150684.064op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.036µs; 6.038µs] or [-0.019%; +0.019%] None None None
credit_card/is_card_number/x371413321323331 throughput [165610589.624op/s; 165673342.969op/s] or [-0.019%; +0.019%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.912µs; 3.913µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ throughput [255550581.431op/s; 255596209.682op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [64.471µs; 64.510µs] or [-0.030%; +0.030%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15501489.372op/s; 15510857.434op/s] or [-0.030%; +0.030%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [58.074µs; 58.119µs] or [-0.038%; +0.038%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [17206313.539op/s; 17219375.750op/s] or [-0.038%; +0.038%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.912µs; 3.913µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255573762.230op/s; 255621753.819op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [54.957µs; 55.037µs] or [-0.073%; +0.073%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [18170204.238op/s; 18196646.255op/s] or [-0.073%; +0.073%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [52.221µs; 52.230µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [19145912.623op/s; 19149435.217op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.038µs; 6.042µs] or [-0.031%; +0.031%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [165508651.057op/s; 165609843.639op/s] or [-0.031%; +0.031%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 89.508µs 89.768µs ± 0.172µs 89.755µs ± 0.054µs 89.809µs 89.889µs 90.182µs 91.783µs 2.26% 8.306 93.889 0.19% 0.012µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [89.745µs; 89.792µs] or [-0.026%; +0.026%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.776µs 26.171µs ± 10.254µs 18.076µs ± 0.217µs 35.235µs 44.280µs 47.884µs 74.225µs 310.62% 0.996 0.972 39.08% 0.725µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.750µs; 27.593µs] or [-5.430%; +5.430%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 185.168µs 185.758µs ± 0.308µs 185.730µs ± 0.221µs 185.958µs 186.210µs 186.309µs 187.891µs 1.16% 1.696 9.794 0.17% 0.022µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 5322231.159op/s 5383353.706op/s ± 8909.612op/s 5384149.294op/s ± 6427.291op/s 5390203.212op/s 5394967.456op/s 5398306.772op/s 5400509.916op/s 0.30% -1.653 9.424 0.17% 630.005op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 17.860µs 17.945µs ± 0.051µs 17.951µs ± 0.037µs 17.981µs 18.018µs 18.036µs 18.174µs 1.24% 0.352 0.795 0.29% 0.004µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 55022769.717op/s 55725961.582op/s ± 159202.072op/s 55707724.509op/s ± 115107.447op/s 55854974.203op/s 55969839.502op/s 55979422.229op/s 55990040.360op/s 0.51% -0.330 0.710 0.28% 11257.286op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.252µs 10.337µs ± 0.033µs 10.336µs ± 0.022µs 10.357µs 10.390µs 10.407µs 10.478µs 1.37% 0.337 0.826 0.32% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 95441303.960op/s 96743963.181op/s ± 312718.887op/s 96752204.446op/s ± 205132.704op/s 96968201.042op/s 97211989.787op/s 97468796.742op/s 97544898.365op/s 0.82% -0.311 0.769 0.32% 22112.565op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [185.716µs; 185.801µs] or [-0.023%; +0.023%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [5382118.919op/s; 5384588.492op/s] or [-0.023%; +0.023%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [17.938µs; 17.952µs] or [-0.040%; +0.040%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [55703897.706op/s; 55748025.458op/s] or [-0.040%; +0.040%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.332µs; 10.341µs] or [-0.045%; +0.045%] None None None
normalization/normalize_name/normalize_name/good throughput [96700623.351op/s; 96787303.011op/s] or [-0.045%; +0.045%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample2_frames_x1000 execution_time 744.767µs 745.688µs ± 0.638µs 745.581µs ± 0.234µs 745.871µs 746.582µs 747.241µs 751.255µs 0.76% 4.235 30.884 0.09% 0.045µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample2_frames_x1000 execution_time [745.600µs; 745.777µs] or [-0.012%; +0.012%] None None None

Group 14

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample_frames_x1000 execution_time 4.183ms 4.187ms ± 0.007ms 4.186ms ± 0.001ms 4.187ms 4.190ms 4.193ms 4.279ms 2.23% 12.099 158.927 0.17% 0.000ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample_frames_x1000 execution_time [4.186ms; 4.188ms] or [-0.023%; +0.023%] None None None

Group 15

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.255µs 3.214µs ± 1.404µs 3.003µs ± 0.033µs 3.038µs 3.660µs 13.821µs 14.829µs 393.76% 7.343 55.187 43.57% 0.099µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.019µs; 3.408µs] or [-6.054%; +6.054%] None None None

Group 16

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching serializing traces from their internal representation to msgpack execution_time 13.928ms 13.992ms ± 0.030ms 13.988ms ± 0.015ms 14.004ms 14.040ms 14.111ms 14.138ms 1.07% 1.797 5.491 0.22% 0.002ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching serializing traces from their internal representation to msgpack execution_time [13.988ms; 13.996ms] or [-0.030%; +0.030%] None None None

Group 17

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
receiver_entry_point/report/2597 execution_time 3.123ms 3.157ms ± 0.021ms 3.150ms ± 0.009ms 3.163ms 3.202ms 3.217ms 3.248ms 3.09% 1.390 1.905 0.67% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
receiver_entry_point/report/2597 execution_time [3.154ms; 3.160ms] or [-0.093%; +0.093%] None None None

Group 18

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 34.649µs 35.015µs ± 0.522µs 34.794µs ± 0.073µs 34.943µs 36.021µs 36.122µs 38.404µs 10.38% 2.418 8.168 1.49% 0.037µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [34.943µs; 35.088µs] or [-0.207%; +0.207%] None None None

Group 19

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 7b48a91 1772814216 cbeauchesne/final_status
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 48.845ms 49.109ms ± 0.947ms 48.960ms ± 0.044ms 49.019ms 49.330ms 51.347ms 60.950ms 24.49% 10.573 123.737 1.92% 0.067ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [48.977ms; 49.240ms] or [-0.267%; +0.267%] None None None

Baseline

Omitted due to size.

@codecov-commenter
Copy link

codecov-commenter commented Mar 6, 2026 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.19%. Comparing base (b70ded5) to head (7b48a91).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1681      +/-   ##
==========================================
+ Coverage   71.17%   71.19%   +0.01%     
==========================================
  Files         427      427              
  Lines       62810    62852      +42     
==========================================
+ Hits        44705    44746      +41     
- Misses      18105    18106       +1
Components Coverage Δ
libdd-crashtracker 63.01% <ø> (+0.01%) ⬆️
libdd-crashtracker-ffi 16.56% <ø> (ø)
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 88.02% <ø> (ø)
libdd-data-pipeline-ffi 75.72% <ø> (ø)
libdd-common 79.73% <ø> (ø)
libdd-common-ffi 73.40% <ø> (ø)
libdd-telemetry 62.48% <ø> (ø)
libdd-telemetry-ffi 16.75% <ø> (ø)
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 80.74% <ø> (ø)
libdd-profiling 81.60% <ø> (ø)
libdd-profiling-ffi 63.65% <ø> (ø)
datadog-sidecar 32.00% <ø> (ø)
datdog-sidecar-ffi 6.43% <ø> (ø)
spawn-worker 54.69% <ø> (ø)
libdd-tinybytes 93.16% <ø> (ø)
libdd-trace-normalization 81.71% <ø> (ø)
libdd-trace-obfuscation 94.69% <ø> (+0.01%) ⬆️
libdd-trace-protobuf 68.00% <ø> (ø)
libdd-trace-utils 89.07% <ø> (ø)
datadog-tracer-flare 88.95% <ø> (ø)
libdd-log 74.69% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dd-octo-sts
Copy link
Contributor

dd-octo-sts bot commented Mar 6, 2026 •
edited
Loading

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 98.65 MB 98.65 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.70 MB 8.70 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.29 MB 11.29 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 114.29 MB 114.29 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 27.16 MB 27.16 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 76.26 KB 76.26 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 186.06 MB 186.02 MB --.02% (-40.00 KB) 💪
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 917.19 MB 917.19 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 9.93 MB 9.93 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 76.26 KB 76.26 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 24.76 MB 24.76 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 51.43 MB 51.43 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 22.97 MB 22.97 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 77.44 KB 77.44 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 190.26 MB 190.23 MB --.02% (-40.00 KB) 💪
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 900.84 MB 900.84 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 7.53 MB 7.53 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 77.44 KB 77.44 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 26.52 MB 26.52 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 47.06 MB 47.06 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 86.54 MB 86.54 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 10.23 MB 10.23 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 107.16 MB 107.16 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.98 MB 11.98 MB 0% (0 B) 👌

@cbeauchesne cbeauchesne marked this pull request as ready for review March 6, 2026 17:50
@cbeauchesne cbeauchesne requested a review from a team as a code owner March 6, 2026 17:50
@danielsn
Copy link
Contributor

danielsn commented Mar 6, 2026

How can we test this?

@cbeauchesne
Copy link
Contributor Author

cbeauchesne commented Mar 6, 2026

Hi @danielsn ,

By looking data in Test Optim.

If you look at results on main, you'll see that the final_status fields is never set : link (all of them are N/A)

If you look over test pushed by this branch for the last commit, all of them are now pass : link

ekump
ekump reviewed Mar 6, 2026
View reviewed changes
.github/workflows/test.yml
report_paths: "target/nextest/ci/junit.xml"
check_name: "[${{ matrix.platform }}:${{ matrix.rust_version }}] test report"
include_passed: true
- name: Add final_status property
Copy link
Contributor

@ekump ekump Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of adding a new step with a new way of modifying the junit.xml file, would it make more sense to incorporate it into the existing script we have that includes unit tests?

Copy link
Contributor Author

@cbeauchesne cbeauchesne Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitly. I confess that since I need to do so for 17 repos (and maybe more), I went for a solution that works everywhere and use it regardless of the existing scripts.

But if you'd like to see an implementation in rust, I can give it a try, but I will very probably need your help, since I never write a single line of rust (Claude, be ready 😄 )

Let me know what you prefer !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ekump ekump ekump left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

ci-build

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@cbeauchesne @codecov-commenter @danielsn @ekump