5 ms·

At the very least, each LINQ method needs an enumerator, which, since it's typed as an interface, cannot be stack-allocated, even for value types (like List<T>'

by ygra 7y ago

At the very least, each LINQ method needs an enumerator, which, since it's typed as an interface, cannot be stack-allocated, even for value types (like List<T>'s enumerator). For Roslyn they noticed that the additional GC pressure was indeed a problem. But for most projects I wouldn't say it's much of a difference (and definitely don't start banning LINQ or other things unless a profiler tells you there's a problem).

jameshart 7y ago
Interestingly, c#’s foreach construct can operate on types that conform the the IEnumerable/IEnumerator interface even if they don’t implement it; you can have a GetEnumerator method return a struct with MoveNext and Current members and it will be stack allocated, so long as the variable on which you foreach is typed with the concrete type rather than IEnumerable. Language integrated query comprehension is also duck-typed, so if you explicitly implemented Select and Where and friends to all return appropriately crafted structs, you could write full LINQ queries and enumerate them without heap allocations. But you’re right that to use LINQ to objects you must have an IEnumerable because you rely on the type to bring in the extension methods. Be interesting to imagine what c# language features would be needed to deliver a stack-allocatable LINQ...
fuzzy2 7y ago
But that’s ~one small-ish object per operator. Unless you’re using `SelectMany` or otherwise generating queries “in a loop” I don’t see how that could be so bad.
- gameswithgo 7y ago
  the gc pressure is normally not so bad, but there is also more indirection in each iteration. So you iterate slower, and evict more data on your cpu caches. Whether this "matters" depends of course. Are you doing a ton of work in each step through the collection? If so, the overhead of linq will be small relative to that. How much do you care about wasting microseconds and eating more battery/power? At scale these little things add up, it can get important quick. On small devices it can be important. For a personal script running on a powerful desktop, it definitely doesn't matter, and everything in between. My LinqFaster repo has some benchmarks showing time differences for some common operations in a fairly worst case scenario (where the work done in each iteration is small) https://github.com/jackmott/LinqFaster https://github.com/jackmott/LinqFaster It would just be nice if C# reliably compiled this abstraction down to efficient code, the way C++, Rust, and Java Streams do.
  - davidgl 7y ago
    There is also http://nessos.github.io/LinqOptimizer/ http://nessos.github.io/LinqOptimizer/

by ygra 7y ago

jameshart 7y ago

Interestingly, c#’s foreach construct can operate on types that conform the the IEnumerable/IEnumerator interface even if they don’t implement it; you can have a GetEnumerator method return a struct with MoveNext and Current members and it will be stack allocated, so long as the variable on which you foreach is typed with the concrete type rather than IEnumerable. Language integrated query comprehension is also duck-typed, so if you explicitly implemented Select and Where and friends to all return appropriately crafted structs, you could write full LINQ queries and enumerate them without heap allocations. But you’re right that to use LINQ to objects you must have an IEnumerable because you rely on the type to bring in the extension methods. Be interesting to imagine what c# language features would be needed to deliver a stack-allocatable LINQ...

fuzzy2 7y ago

But that’s ~one small-ish object per operator. Unless you’re using `SelectMany` or otherwise generating queries “in a loop” I don’t see how that could be so bad.

gameswithgo 7y ago
the gc pressure is normally not so bad, but there is also more indirection in each iteration. So you iterate slower, and evict more data on your cpu caches. Whether this "matters" depends of course. Are you doing a ton of work in each step through the collection? If so, the overhead of linq will be small relative to that. How much do you care about wasting microseconds and eating more battery/power? At scale these little things add up, it can get important quick. On small devices it can be important. For a personal script running on a powerful desktop, it definitely doesn't matter, and everything in between. My LinqFaster repo has some benchmarks showing time differences for some common operations in a fairly worst case scenario (where the work done in each iteration is small) https://github.com/jackmott/LinqFaster https://github.com/jackmott/LinqFaster It would just be nice if C# reliably compiled this abstraction down to efficient code, the way C++, Rust, and Java Streams do.
- davidgl 7y ago
  There is also http://nessos.github.io/LinqOptimizer/ http://nessos.github.io/LinqOptimizer/