Add a "ChunkAsync" extension method for IAsyncEnumerable

[JasonWei512]

An async version of .NET 6's IEnumerable.Chunk(int size) (Link).

It splits an IAsyncEnumerable<T> into T[]s of size.

[JKamsker]

Neat idea but atleast one alloc would be wasted (The list won't be reused && the growth of the list)

[JKamsker]

IMHO this solution is a bit too complicated. Do we really need to return an Array? If not, we could just use a List or wrap it in a ReadOnlyList.

This is what i came up with:

public static async IAsyncEnumerable<IReadOnlyCollection<TSource>> ChunkAsync<TSource>
(
    this IAsyncEnumerable<TSource> source, 
    int size, 
    [EnumeratorCancellation] CancellationToken cancellationToken = default
)
{
    if (source == null)
        throw new ArgumentNullException(nameof(source));

    if (size < 1)
        throw new ArgumentOutOfRangeException(nameof(size));

    await using var e = source.GetConfiguredAsyncEnumerator(cancellationToken, false);
    var buffer = new List<TSource>(size);
    while (await e.MoveNextAsync())
    {
        buffer.Add(e.Current);
        if (buffer.Count >= size)
        {
            yield return buffer;
            buffer = new List<TSource>(size);
        }
    }

    if (buffer.Count > 0)
    {
        yield return buffer;
    }
}

Now the list is initialized with a capacity of the length - no idea if thats good or not....

[JasonWei512]

Is this repository still maintained?
Maybe Microsoft should add IAsyncEnumerable Linq to .NET BCL.

[theodorzoulias]

@JKamsker apparently some people like to turn off chunking conditionally by passing a large size, like int.MaxValue, in which case initializing the Capacity of the List results in OutOfMemoryException. Microsoft had to fix many subtle bugs like this (including performance regressions), before settling on the current complex implementation.

[JKamsker]

@theodorzoulias Lol yea then it might be the better option :D