Skip to content

Adapt dotnet memory allocations#705

Open
ricardoboss wants to merge 13 commits intobencherdev:develfrom
ricardoboss:issues/699-dotnet-memory-allocations
Open

Adapt dotnet memory allocations#705
ricardoboss wants to merge 13 commits intobencherdev:develfrom
ricardoboss:issues/699-dotnet-memory-allocations

Conversation

@ricardoboss
Copy link

Fixes #699

Adds handling for the memory key in BenchmarkDotNet, in case C# benchmarks are run with the [MemoryDiagnoser] attribute.
If found, adds a new metric allocated measured in bytes.

@ricardoboss ricardoboss force-pushed the issues/699-dotnet-memory-allocations branch from 2841ffc to 8edf525 Compare March 14, 2026 20:31
Copy link
Member

@epompeii epompeii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ricardoboss thank you for all of your work so far!
I've made a few suggestions and had a couple of questions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to get a smaller example file or are they all this large?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah of course, it was just easily accessible for me. I can provide a smaller one

pub mod dotnet {
use bencher_valid::{BYTES, NANOSECONDS};

create_measure!(Latency, "Latency", "latency", NANOSECONDS);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't need to create a new Latency Measure, we should just use the existing one.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh ok, I thought every adapter should define its own set. Will switch to the default measure then

Some(results_map.into())
}

pub fn new_dotnet(benchmark_metrics: Vec<(BenchmarkName, Vec<DotNetMeasure>)>) -> Option<Self> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency, lets move this method above new_iai. Adapters are (mostly) sorted alphabetically across the code base.

}

#[test]
fn adapter_c_sharp_dot_net_memory() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the harness collect both memory and wall-clock time measurements at the same time or are they mutually exclusive?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is above my paygrade... I have no idea 😅

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like it does:

Yes, BenchmarkDotNet collects both simultaneously (well, technically in separate runs to avoid interference, but in the same invocation). When you add [MemoryDiagnoser], the harness:

  1. Runs timing measurements normally
  2. Performs a separate run for memory/GC metrics (so they don't skew latency numbers)
  3. Combines everything into one unified JSON output per benchmark

In the sample file, there are both:

{                                                                                                                                                                                                                                                                                        
    "Method": "Tokenize",                                                                                                                                                                                                                                                                  
    "Parameters": "ExampleFileName=expressions.step",                                                                                                                                                                                                                                      
    "Statistics": {                                                                                                                                                                                                                                                                        
      "Mean": 106.338...,                                                                                                                                                                                                                                                                  
      "Median": 105.607...,
      "StandardDeviation": 3.223...,
      ...
    },
    "Memory": {
      "Gen0Collections": 168,
      "Gen1Collections": 0,
      "Gen2Collections": 0,
      "TotalOperations": 4194304,
      "BytesAllocatedPerOperation": 672
    }
  }

With that being the case, we're going to want to make sure we:

  1. Always collect the wall-clock time measurements
  2. Optionally collect the memory measurements (if they are present)

Right now, it seems like we are only collecting the memory measurements if they are present (skipping 1.).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I follow... How is the statistics object relevant for the memory allocations? Isn't this used for the latency measurement? That part should mostly be untouched by my changes (latency is still being collected)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Then we just need to make sure we are testing that the latency measures are still being collected properly.

That is, this adapter_c_sharp_dot_net_memory test should assert on each expected measure (including Latency).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, no problem

);
}

pub mod dotnet {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency, lets move this below json and above iai.

@ricardoboss ricardoboss requested a review from epompeii March 15, 2026 22:26
@ricardoboss ricardoboss marked this pull request as ready for review March 15, 2026 22:27
if memory.is_some() {
let json_allocated_metric = JsonNewMetric {
value: memory.unwrap().bytes_allocated_per_operation.into(),
let m = memory.unwrap();
Copy link
Member

@epompeii epompeii Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never unwrap in production code. Try this instead:

Suggested change
let m = memory.unwrap();
if let Some(memory) = memory {

Note this would replace the if memory.is_some() { above, so we wouldn't need the let m = memory.unwrap(); at all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

c_sharp_dot_net adapter should support memory allocations

2 participants