Conversation
BenchmarksComparisonBenchmark execution time: 2026-03-11 13:12:57 Comparing candidate commit 72f61f6 in PR branch Found 5 performance improvements and 13 performance regressions! Performance is the same for 40 metrics, 2 unstable metrics.
|
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| normalization/normalize_trace/test_trace | execution_time | 245.915ns | 255.009ns ± 11.509ns | 249.739ns ± 2.870ns | 257.722ns | 282.757ns | 284.319ns | 285.576ns | 14.35% | 1.473 | 0.772 | 4.50% | 0.814ns | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| normalization/normalize_trace/test_trace | execution_time | [253.414ns; 256.604ns] or [-0.625%; +0.625%] | None | None | None |
Group 2
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| benching serializing traces from their internal representation to msgpack | execution_time | 14.859ms | 14.911ms ± 0.031ms | 14.907ms ± 0.012ms | 14.920ms | 14.963ms | 15.037ms | 15.080ms | 1.16% | 2.426 | 8.324 | 0.21% | 0.002ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| benching serializing traces from their internal representation to msgpack | execution_time | [14.907ms; 14.916ms] or [-0.029%; +0.029%] | None | None | None |
Group 3
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| single_flag_killswitch/rules-based | execution_time | 190.385ns | 192.889ns ± 2.038ns | 192.549ns ± 1.451ns | 194.051ns | 196.920ns | 199.119ns | 199.578ns | 3.65% | 0.940 | 0.566 | 1.05% | 0.144ns | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| single_flag_killswitch/rules-based | execution_time | [192.607ns; 193.171ns] or [-0.146%; +0.146%] | None | None | None |
Group 4
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| profile_add_sample2_frames_x1000 | execution_time | 733.712µs | 735.201µs ± 0.606µs | 735.138µs ± 0.458µs | 735.652µs | 736.249µs | 736.481µs | 736.553µs | 0.19% | 0.124 | -0.546 | 0.08% | 0.043µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| profile_add_sample2_frames_x1000 | execution_time | [735.117µs; 735.285µs] or [-0.011%; +0.011%] | None | None | None |
Group 5
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| receiver_entry_point/report/2598 | execution_time | 3.581ms | 3.616ms ± 0.018ms | 3.613ms ± 0.010ms | 3.624ms | 3.653ms | 3.663ms | 3.686ms | 2.02% | 1.001 | 1.138 | 0.50% | 0.001ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| receiver_entry_point/report/2598 | execution_time | [3.613ms; 3.618ms] or [-0.069%; +0.069%] | None | None | None |
Group 6
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| profile_add_sample_frames_x1000 | execution_time | 4.130ms | 4.137ms ± 0.007ms | 4.136ms ± 0.002ms | 4.138ms | 4.142ms | 4.150ms | 4.219ms | 2.01% | 9.731 | 117.258 | 0.16% | 0.000ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| profile_add_sample_frames_x1000 | execution_time | [4.136ms; 4.138ms] or [-0.022%; +0.022%] | None | None | None |
Group 7
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| sdk_test_data/rules-based | execution_time | 144.023µs | 146.338µs ± 1.687µs | 146.075µs ± 0.503µs | 146.610µs | 147.813µs | 151.912µs | 162.375µs | 11.16% | 5.958 | 47.783 | 1.15% | 0.119µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| sdk_test_data/rules-based | execution_time | [146.104µs; 146.571µs] or [-0.160%; +0.160%] | None | None | None |
Group 8
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ip_address/quantize_peer_ip_address_benchmark | execution_time | 5.008µs | 5.089µs ± 0.043µs | 5.083µs ± 0.029µs | 5.114µs | 5.162µs | 5.166µs | 5.167µs | 1.65% | 0.123 | -0.868 | 0.84% | 0.003µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| ip_address/quantize_peer_ip_address_benchmark | execution_time | [5.083µs; 5.095µs] or [-0.116%; +0.116%] | None | None | None |
Group 9
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| benching string interning on wordpress profile | execution_time | 162.063µs | 162.756µs ± 0.392µs | 162.680µs ± 0.126µs | 162.840µs | 163.311µs | 163.820µs | 166.614µs | 2.42% | 5.128 | 45.576 | 0.24% | 0.028µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| benching string interning on wordpress profile | execution_time | [162.702µs; 162.811µs] or [-0.033%; +0.033%] | None | None | None |
Group 10
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| tags/replace_trace_tags | execution_time | 2.399µs | 2.445µs ± 0.018µs | 2.446µs ± 0.007µs | 2.452µs | 2.476µs | 2.506µs | 2.509µs | 2.59% | 0.533 | 2.017 | 0.73% | 0.001µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| tags/replace_trace_tags | execution_time | [2.442µs; 2.447µs] or [-0.102%; +0.102%] | None | None | None |
Group 11
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| benching deserializing traces from msgpack to their internal representation | execution_time | 49.756ms | 50.033ms ± 1.060ms | 49.882ms ± 0.035ms | 49.923ms | 50.068ms | 55.637ms | 59.784ms | 19.85% | 8.299 | 69.746 | 2.11% | 0.075ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| benching deserializing traces from msgpack to their internal representation | execution_time | [49.886ms; 50.180ms] or [-0.294%; +0.294%] | None | None | None |
Group 12
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| two way interface | execution_time | 17.584µs | 26.104µs ± 9.997µs | 18.595µs ± 0.800µs | 34.815µs | 43.783µs | 53.629µs | 69.632µs | 274.48% | 1.102 | 1.293 | 38.20% | 0.707µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| two way interface | execution_time | [24.719µs; 27.490µs] or [-5.308%; +5.308%] | None | None | None |
Group 13
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | execution_time | 493.229µs | 494.027µs ± 0.440µs | 493.981µs ± 0.239µs | 494.245µs | 494.665µs | 494.914µs | 497.640µs | 0.74% | 2.918 | 21.313 | 0.09% | 0.031µs | 1 | 200 |
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | throughput | 2009484.095op/s | 2024181.921op/s ± 1796.718op/s | 2024371.066op/s ± 978.874op/s | 2025346.594op/s | 2026460.429op/s | 2026994.013op/s | 2027454.665op/s | 0.15% | -2.879 | 20.902 | 0.09% | 127.047op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | execution_time | 370.029µs | 370.727µs ± 0.283µs | 370.699µs ± 0.210µs | 370.928µs | 371.235µs | 371.348µs | 371.566µs | 0.23% | 0.328 | -0.293 | 0.08% | 0.020µs | 1 | 200 |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | throughput | 2691315.371op/s | 2697407.614op/s ± 2057.268op/s | 2697603.092op/s ± 1531.354op/s | 2699040.919op/s | 2700716.518op/s | 2701349.904op/s | 2702489.616op/s | 0.18% | -0.324 | -0.296 | 0.08% | 145.471op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | execution_time | 167.242µs | 167.608µs ± 0.240µs | 167.584µs ± 0.096µs | 167.684µs | 167.820µs | 167.942µs | 169.733µs | 1.28% | 6.044 | 49.092 | 0.14% | 0.017µs | 1 | 200 |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | throughput | 5891590.084op/s | 5966299.442op/s ± 8463.279op/s | 5967150.145op/s ± 3404.724op/s | 5970292.864op/s | 5973353.758op/s | 5976774.004op/s | 5979349.274op/s | 0.20% | -5.982 | 48.398 | 0.14% | 598.444op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/[empty string] | execution_time | 37.130µs | 37.202µs ± 0.029µs | 37.201µs ± 0.021µs | 37.221µs | 37.248µs | 37.277µs | 37.292µs | 0.25% | 0.283 | 0.083 | 0.08% | 0.002µs | 1 | 200 |
| normalization/normalize_service/normalize_service/[empty string] | throughput | 26815313.942op/s | 26880282.896op/s ± 20839.459op/s | 26881257.107op/s ± 15059.942op/s | 26896259.466op/s | 26910440.006op/s | 26929052.798op/s | 26932630.085op/s | 0.19% | -0.279 | 0.078 | 0.08% | 1473.572op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/test_ASCII | execution_time | 45.373µs | 45.605µs ± 0.111µs | 45.600µs ± 0.094µs | 45.701µs | 45.764µs | 45.793µs | 45.876µs | 0.61% | -0.024 | -0.872 | 0.24% | 0.008µs | 1 | 200 |
| normalization/normalize_service/normalize_service/test_ASCII | throughput | 21797932.200op/s | 21927498.619op/s ± 53413.831op/s | 21929840.592op/s ± 45041.443op/s | 21970648.803op/s | 22011402.177op/s | 22032228.740op/s | 22039565.919op/s | 0.50% | 0.032 | -0.873 | 0.24% | 3776.928op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | execution_time | [493.966µs; 494.088µs] or [-0.012%; +0.012%] | None | None | None |
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | throughput | [2023932.913op/s; 2024430.928op/s] or [-0.012%; +0.012%] | None | None | None |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | execution_time | [370.687µs; 370.766µs] or [-0.011%; +0.011%] | None | None | None |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | throughput | [2697122.497op/s; 2697692.732op/s] or [-0.011%; +0.011%] | None | None | None |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | execution_time | [167.575µs; 167.642µs] or [-0.020%; +0.020%] | None | None | None |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | throughput | [5965126.513op/s; 5967472.371op/s] or [-0.020%; +0.020%] | None | None | None |
| normalization/normalize_service/normalize_service/[empty string] | execution_time | [37.198µs; 37.206µs] or [-0.011%; +0.011%] | None | None | None |
| normalization/normalize_service/normalize_service/[empty string] | throughput | [26877394.748op/s; 26883171.045op/s] or [-0.011%; +0.011%] | None | None | None |
| normalization/normalize_service/normalize_service/test_ASCII | execution_time | [45.590µs; 45.621µs] or [-0.034%; +0.034%] | None | None | None |
| normalization/normalize_service/normalize_service/test_ASCII | throughput | [21920095.975op/s; 21934901.262op/s] or [-0.034%; +0.034%] | None | None | None |
Group 14
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| credit_card/is_card_number/ | execution_time | 3.891µs | 3.911µs ± 0.002µs | 3.911µs ± 0.001µs | 3.912µs | 3.914µs | 3.915µs | 3.916µs | 0.12% | -2.789 | 22.688 | 0.06% | 0.000µs | 1 | 200 |
| credit_card/is_card_number/ | throughput | 255378714.344op/s | 255692522.139op/s ± 157913.384op/s | 255686179.027op/s ± 81745.182op/s | 255776799.318op/s | 255897897.140op/s | 255995958.745op/s | 257014918.529op/s | 0.52% | 2.820 | 23.011 | 0.06% | 11166.162op/s | 1 | 200 |
| credit_card/is_card_number/ 3782-8224-6310-005 | execution_time | 79.367µs | 80.003µs ± 0.328µs | 79.947µs ± 0.198µs | 80.186µs | 80.630µs | 80.952µs | 81.026µs | 1.35% | 0.741 | 0.386 | 0.41% | 0.023µs | 1 | 200 |
| credit_card/is_card_number/ 3782-8224-6310-005 | throughput | 12341693.103op/s | 12499808.209op/s ± 51061.495op/s | 12508360.590op/s ± 30992.290op/s | 12533940.348op/s | 12573229.319op/s | 12581957.513op/s | 12599711.598op/s | 0.73% | -0.719 | 0.340 | 0.41% | 3610.593op/s | 1 | 200 |
| credit_card/is_card_number/ 378282246310005 | execution_time | 72.902µs | 73.387µs ± 0.287µs | 73.338µs ± 0.187µs | 73.562µs | 73.928µs | 74.175µs | 74.464µs | 1.54% | 0.921 | 0.824 | 0.39% | 0.020µs | 1 | 200 |
| credit_card/is_card_number/ 378282246310005 | throughput | 13429384.473op/s | 13626544.506op/s ± 53129.710op/s | 13635559.034op/s ± 34910.721op/s | 13666475.460op/s | 13691563.401op/s | 13710917.497op/s | 13717090.123op/s | 0.60% | -0.898 | 0.757 | 0.39% | 3756.838op/s | 1 | 200 |
| credit_card/is_card_number/37828224631 | execution_time | 3.893µs | 3.911µs ± 0.003µs | 3.911µs ± 0.001µs | 3.912µs | 3.915µs | 3.918µs | 3.922µs | 0.28% | -0.586 | 10.460 | 0.07% | 0.000µs | 1 | 200 |
| credit_card/is_card_number/37828224631 | throughput | 254976021.086op/s | 255677935.619op/s ± 174427.891op/s | 255698542.520op/s ± 83165.760op/s | 255775031.612op/s | 255877575.216op/s | 255929071.108op/s | 256845660.067op/s | 0.45% | 0.611 | 10.593 | 0.07% | 12333.914op/s | 1 | 200 |
| credit_card/is_card_number/378282246310005 | execution_time | 69.730µs | 70.217µs ± 0.253µs | 70.196µs ± 0.177µs | 70.382µs | 70.705µs | 70.838µs | 70.940µs | 1.06% | 0.467 | -0.181 | 0.36% | 0.018µs | 1 | 200 |
| credit_card/is_card_number/378282246310005 | throughput | 14096329.188op/s | 14241676.911op/s ± 51238.175op/s | 14245886.292op/s ± 35841.755op/s | 14279677.520op/s | 14316621.457op/s | 14336879.277op/s | 14341062.765op/s | 0.67% | -0.449 | -0.200 | 0.36% | 3623.086op/s | 1 | 200 |
| credit_card/is_card_number/37828224631000521389798 | execution_time | 52.121µs | 52.194µs ± 0.039µs | 52.188µs ± 0.026µs | 52.218µs | 52.258µs | 52.319µs | 52.339µs | 0.29% | 0.864 | 1.142 | 0.07% | 0.003µs | 1 | 200 |
| credit_card/is_card_number/37828224631000521389798 | throughput | 19106340.718op/s | 19159464.723op/s ± 14321.886op/s | 19161416.572op/s ± 9419.871op/s | 19169491.276op/s | 19179182.724op/s | 19183427.916op/s | 19185998.059op/s | 0.13% | -0.858 | 1.126 | 0.07% | 1012.710op/s | 1 | 200 |
| credit_card/is_card_number/x371413321323331 | execution_time | 5.700µs | 5.867µs ± 0.067µs | 5.865µs ± 0.046µs | 5.911µs | 5.986µs | 6.017µs | 6.027µs | 2.77% | 0.089 | -0.330 | 1.13% | 0.005µs | 1 | 200 |
| credit_card/is_card_number/x371413321323331 | throughput | 165922710.025op/s | 170463431.287op/s ± 1932935.667op/s | 170511843.947op/s ± 1327983.603op/s | 171807278.247op/s | 173459522.716op/s | 175278223.219op/s | 175435578.315op/s | 2.89% | -0.032 | -0.323 | 1.13% | 136679.192op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/ | execution_time | 3.892µs | 3.912µs ± 0.003µs | 3.911µs ± 0.001µs | 3.913µs | 3.916µs | 3.920µs | 3.927µs | 0.39% | -0.347 | 10.542 | 0.08% | 0.000µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/ | throughput | 254679448.514op/s | 255647751.731op/s ± 200910.978op/s | 255661778.564op/s ± 92070.409op/s | 255753701.884op/s | 255895616.848op/s | 255950764.743op/s | 256957007.987op/s | 0.51% | 0.376 | 10.658 | 0.08% | 14206.551op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | execution_time | 64.087µs | 64.354µs ± 0.146µs | 64.319µs ± 0.059µs | 64.395µs | 64.695µs | 64.818µs | 64.936µs | 0.96% | 1.662 | 2.720 | 0.23% | 0.010µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | throughput | 15399845.133op/s | 15539096.150op/s ± 35230.409op/s | 15547416.306op/s ± 14346.112op/s | 15560979.677op/s | 15576528.900op/s | 15583025.398op/s | 15603896.293op/s | 0.36% | -1.649 | 2.673 | 0.23% | 2491.166op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 378282246310005 | execution_time | 57.368µs | 57.562µs ± 0.123µs | 57.541µs ± 0.071µs | 57.618µs | 57.779µs | 57.890µs | 58.187µs | 1.12% | 1.602 | 4.805 | 0.21% | 0.009µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 378282246310005 | throughput | 17185891.292op/s | 17372661.679op/s ± 37099.571op/s | 17378856.885op/s ± 21288.329op/s | 17398139.704op/s | 17419587.804op/s | 17428243.079op/s | 17431231.538op/s | 0.30% | -1.575 | 4.653 | 0.21% | 2623.336op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631 | execution_time | 3.895µs | 3.912µs ± 0.003µs | 3.912µs ± 0.002µs | 3.914µs | 3.917µs | 3.920µs | 3.924µs | 0.30% | -0.305 | 4.737 | 0.08% | 0.000µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631 | throughput | 254868466.142op/s | 255631653.206op/s ± 200503.649op/s | 255635196.478op/s ± 128424.805op/s | 255773189.545op/s | 255896168.727op/s | 255959243.969op/s | 256756243.836op/s | 0.44% | 0.320 | 4.796 | 0.08% | 14177.749op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/378282246310005 | execution_time | 54.382µs | 54.791µs ± 0.212µs | 54.739µs ± 0.104µs | 54.877µs | 55.207µs | 55.467µs | 55.574µs | 1.52% | 1.132 | 1.525 | 0.39% | 0.015µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/378282246310005 | throughput | 17994171.069op/s | 18251385.120op/s ± 70289.480op/s | 18268403.885op/s ± 34629.211op/s | 18296273.964op/s | 18336347.684op/s | 18377051.255op/s | 18388396.701op/s | 0.66% | -1.106 | 1.450 | 0.38% | 4970.217op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | execution_time | 52.122µs | 52.193µs ± 0.035µs | 52.190µs ± 0.026µs | 52.217µs | 52.245µs | 52.297µs | 52.306µs | 0.22% | 0.433 | 0.053 | 0.07% | 0.002µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | throughput | 19118364.583op/s | 19159783.103op/s ± 12787.690op/s | 19160657.423op/s ± 9616.162op/s | 19170041.538op/s | 19179035.588op/s | 19183797.883op/s | 19185637.632op/s | 0.13% | -0.429 | 0.045 | 0.07% | 904.226op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/x371413321323331 | execution_time | 5.721µs | 5.859µs ± 0.062µs | 5.861µs ± 0.036µs | 5.900µs | 5.965µs | 5.994µs | 6.004µs | 2.45% | 0.029 | -0.524 | 1.06% | 0.004µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/x371413321323331 | throughput | 166553873.442op/s | 170692294.398op/s ± 1810679.255op/s | 170628571.761op/s ± 1056451.852op/s | 171675132.816op/s | 173603964.137op/s | 174589926.186op/s | 174800938.515op/s | 2.45% | 0.018 | -0.526 | 1.06% | 128034.358op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| credit_card/is_card_number/ | execution_time | [3.911µs; 3.911µs] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number/ | throughput | [255670636.863op/s; 255714407.416op/s] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number/ 3782-8224-6310-005 | execution_time | [79.957µs; 80.048µs] or [-0.057%; +0.057%] | None | None | None |
| credit_card/is_card_number/ 3782-8224-6310-005 | throughput | [12492731.577op/s; 12506884.841op/s] or [-0.057%; +0.057%] | None | None | None |
| credit_card/is_card_number/ 378282246310005 | execution_time | [73.347µs; 73.427µs] or [-0.054%; +0.054%] | None | None | None |
| credit_card/is_card_number/ 378282246310005 | throughput | [13619181.239op/s; 13633907.773op/s] or [-0.054%; +0.054%] | None | None | None |
| credit_card/is_card_number/37828224631 | execution_time | [3.911µs; 3.912µs] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number/37828224631 | throughput | [255653761.591op/s; 255702109.647op/s] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number/378282246310005 | execution_time | [70.182µs; 70.252µs] or [-0.050%; +0.050%] | None | None | None |
| credit_card/is_card_number/378282246310005 | throughput | [14234575.793op/s; 14248778.030op/s] or [-0.050%; +0.050%] | None | None | None |
| credit_card/is_card_number/37828224631000521389798 | execution_time | [52.188µs; 52.199µs] or [-0.010%; +0.010%] | None | None | None |
| credit_card/is_card_number/37828224631000521389798 | throughput | [19157479.847op/s; 19161449.599op/s] or [-0.010%; +0.010%] | None | None | None |
| credit_card/is_card_number/x371413321323331 | execution_time | [5.858µs; 5.876µs] or [-0.157%; +0.157%] | None | None | None |
| credit_card/is_card_number/x371413321323331 | throughput | [170195544.994op/s; 170731317.580op/s] or [-0.157%; +0.157%] | None | None | None |
| credit_card/is_card_number_no_luhn/ | execution_time | [3.911µs; 3.912µs] or [-0.011%; +0.011%] | None | None | None |
| credit_card/is_card_number_no_luhn/ | throughput | [255619907.402op/s; 255675596.060op/s] or [-0.011%; +0.011%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | execution_time | [64.334µs; 64.374µs] or [-0.032%; +0.032%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | throughput | [15534213.554op/s; 15543978.746op/s] or [-0.031%; +0.031%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 378282246310005 | execution_time | [57.545µs; 57.579µs] or [-0.030%; +0.030%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 378282246310005 | throughput | [17367520.035op/s; 17377803.323op/s] or [-0.030%; +0.030%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631 | execution_time | [3.911µs; 3.912µs] or [-0.011%; +0.011%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631 | throughput | [255603865.328op/s; 255659441.083op/s] or [-0.011%; +0.011%] | None | None | None |
| credit_card/is_card_number_no_luhn/378282246310005 | execution_time | [54.762µs; 54.821µs] or [-0.054%; +0.054%] | None | None | None |
| credit_card/is_card_number_no_luhn/378282246310005 | throughput | [18241643.674op/s; 18261126.566op/s] or [-0.053%; +0.053%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | execution_time | [52.188µs; 52.198µs] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | throughput | [19158010.853op/s; 19161555.354op/s] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number_no_luhn/x371413321323331 | execution_time | [5.851µs; 5.868µs] or [-0.147%; +0.147%] | None | None | None |
| credit_card/is_card_number_no_luhn/x371413321323331 | throughput | [170441351.667op/s; 170943237.128op/s] or [-0.147%; +0.147%] | None | None | None |
Group 15
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| concentrator/add_spans_to_concentrator | execution_time | 10.645ms | 10.681ms ± 0.017ms | 10.679ms ± 0.009ms | 10.689ms | 10.708ms | 10.727ms | 10.772ms | 0.87% | 1.504 | 5.806 | 0.16% | 0.001ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| concentrator/add_spans_to_concentrator | execution_time | [10.679ms; 10.684ms] or [-0.022%; +0.022%] | None | None | None |
Group 16
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | execution_time | 205.907µs | 206.679µs ± 0.470µs | 206.587µs ± 0.311µs | 206.959µs | 207.564µs | 208.015µs | 208.236µs | 0.80% | 0.818 | 0.287 | 0.23% | 0.033µs | 1 | 200 |
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | throughput | 4802249.218op/s | 4838447.370op/s ± 10985.810op/s | 4840585.457op/s ± 7304.463op/s | 4847180.041op/s | 4851997.615op/s | 4854394.653op/s | 4856569.107op/s | 0.33% | -0.807 | 0.258 | 0.23% | 776.814op/s | 1 | 200 |
| normalization/normalize_name/normalize_name/bad-name | execution_time | 18.588µs | 18.684µs ± 0.107µs | 18.641µs ± 0.020µs | 18.683µs | 18.876µs | 19.111µs | 19.210µs | 3.05% | 2.439 | 6.361 | 0.57% | 0.008µs | 1 | 200 |
| normalization/normalize_name/normalize_name/bad-name | throughput | 52056519.263op/s | 53523056.355op/s ± 301512.958op/s | 53643837.218op/s ± 58274.034op/s | 53681553.746op/s | 53752497.129op/s | 53787974.351op/s | 53798173.871op/s | 0.29% | -2.398 | 6.101 | 0.56% | 21320.186op/s | 1 | 200 |
| normalization/normalize_name/normalize_name/good | execution_time | 10.632µs | 10.726µs ± 0.045µs | 10.718µs ± 0.025µs | 10.748µs | 10.801µs | 10.865µs | 10.886µs | 1.57% | 0.797 | 1.173 | 0.42% | 0.003µs | 1 | 200 |
| normalization/normalize_name/normalize_name/good | throughput | 91858658.275op/s | 93236899.206op/s ± 388883.443op/s | 93303314.886op/s ± 221746.812op/s | 93460044.562op/s | 93733503.968op/s | 94024354.000op/s | 94052574.671op/s | 0.80% | -0.765 | 1.102 | 0.42% | 27498.212op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | execution_time | [206.614µs; 206.744µs] or [-0.032%; +0.032%] | None | None | None |
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | throughput | [4836924.842op/s; 4839969.897op/s] or [-0.031%; +0.031%] | None | None | None |
| normalization/normalize_name/normalize_name/bad-name | execution_time | [18.669µs; 18.699µs] or [-0.079%; +0.079%] | None | None | None |
| normalization/normalize_name/normalize_name/bad-name | throughput | [53481269.559op/s; 53564843.151op/s] or [-0.078%; +0.078%] | None | None | None |
| normalization/normalize_name/normalize_name/good | execution_time | [10.719µs; 10.732µs] or [-0.058%; +0.058%] | None | None | None |
| normalization/normalize_name/normalize_name/good | throughput | [93183003.701op/s; 93290794.711op/s] or [-0.058%; +0.058%] | None | None | None |
Group 17
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| sql/obfuscate_sql_string | execution_time | 85.304µs | 85.509µs ± 0.166µs | 85.488µs ± 0.037µs | 85.535µs | 85.603µs | 85.690µs | 87.668µs | 2.55% | 11.151 | 142.798 | 0.19% | 0.012µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| sql/obfuscate_sql_string | execution_time | [85.486µs; 85.532µs] or [-0.027%; +0.027%] | None | None | None |
Group 18
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| write only interface | execution_time | 1.242µs | 3.257µs ± 1.470µs | 3.006µs ± 0.028µs | 3.044µs | 3.687µs | 14.380µs | 15.517µs | 416.21% | 7.289 | 54.686 | 45.01% | 0.104µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| write only interface | execution_time | [3.053µs; 3.460µs] or [-6.254%; +6.254%] | None | None | None |
Group 19
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 72f61f6 | 1773233726 | vianney/implement-shared-runtime |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| redis/obfuscate_redis_string | execution_time | 32.996µs | 33.734µs ± 1.225µs | 33.174µs ± 0.100µs | 33.305µs | 36.378µs | 36.412µs | 36.685µs | 10.58% | 1.676 | 0.861 | 3.62% | 0.087µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| redis/obfuscate_redis_string | execution_time | [33.564µs; 33.903µs] or [-0.503%; +0.503%] | None | None | None |
Baseline
Omitted due to size.
f615fad to
ff6448c
Compare
📚 Documentation Check Results📦
|
Clippy Allow Annotation ReportComparing clippy allow annotations between branches:
Summary by Rule
Annotation Counts by File
Annotation Stats by Crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. |
🔒 Cargo Deny Results📦
|
With the refactor the stop action is processed right away without processing all actions sent before it. This can cause some message to be lost if they were submitted right before shutdown.
|
✅ Tests 🎉 All green!❄️ No new flaky tests detected 🔗 Commit SHA: 72f61f6 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback! |
| /// | ||
| /// # Errors | ||
| /// Returns an error if it fails to create a runtime. | ||
| pub fn runtime(&self) -> Result<Arc<Runtime>, io::Error> { |
There was a problem hiding this comment.
Is returning a single-threaded runtime ok?
When would the underlying runtime not be available? In between the before_fork() and after_fork_* functions?
There was a problem hiding this comment.
If a component blocking function (e.g. TraceExporter::send) requires a tokio runtime while we are preparing for a fork, we want to allow the component to access one to avoid losing data (e.g. the trace payload) without restarting the full runtime which we would have to shutdown again. The components should use the runtime with block_on so a current thread runtime will behave in the same way as the multi-threaded one. Since the SharedRuntime doesn't hold a reference to this new runtime, it will be dropped when the blocking function returns guarantying no runtime is running across the fork.
What does this PR do?
This PR introduces a shared runtime to be used across common components it offers two main features:
TraceExporter::sendThis PR also refactors the
Workertrait to make it execute a trigger-run loop. The worker can be paused between two calls torunbut not within arunexecution. This is the same tradeoff between fork-time impact and worker execution than the one in dd-trace-py.Motivation
Be able to share a common tokio runtime across components and manage fork-safety in a single place.
Additional Notes
This PR is a breaking change especially for language which register "fork-safety" hooks as this should now be handled in the SharedRuntime.
How to test the change?
Describe here in detail how the change can be validated.