4 ms·

One of the tips is to avoid Linq, which many .NET developers are hesitant to do. I made a library that lets you use Linq style convenience functions without a

One of the tips is to avoid Linq, which many .NET developers are hesitant to do. I made a library that lets you use Linq style convenience functions without a performance hit in many cases: https://github.com/jackmott/LinqFaster https://github.com/jackmott/LinqFaster

GordonS 8y ago
This looks really interesting - are the benchmarks in the GitHub repo against .NET Framework? With all the performance work that's been done in .NET Core, it might be interesting to compare it with .NET Core. Also, I noticed there haven't been any commits for a while - is this more or less considered 'complete'?
- gameswithgo 8y ago
  Yes it is more or less considered complete. I may do a pass over it to see if it can leverage or support span<t> better sometime soon. Performance deltas should be similar on .NET Core.
darrenkopp 8y ago
Also worth mentioning: https://github.com/kevin-montrose/LinqAF https://github.com/kevin-montrose/LinqAF Focused on reducing the allocations which makes linq heavy
WorldMaker 8y ago
So your library speeds up Array<T>/T[] and List<T> Linq operations? That's quite interesting, though usually in my experiences in Linq performance optimization the problem is usually prematurely using the wrong data type for your Linq operation. Given the immediate next bullet point in the article is about foreach operations over List<T>, that would be my top expectation why they were seeing so much allocation in Linq. ToList() allocates a lot, and to many projects I've seen reify everything with ToList() way too often in Linq usage. I've argued before that List<T> is very rarely the appropriate data structure for a lot of Linq work, and personally consider ToList() harmful. A successful strategy I've used to cleaning up Linq performance in projects is to simply start by remove all ToList() calls entirely and work to move API signatures to use smarter, more appropriate data types than List<T> and IList<T> everywhere.
- caseymarquis 8y ago
  You just need to be careful not to allow any subsequent modifications of the original collection when doing that (assuming you're using the returned LINQ result), or you'll introduce runtime errors. Typically, my use of ToList or ToArray means I'm guarding against future runtime errors due to collection modification at the expense of memory allocation. It sounds like you know this already, but I like to think I just saved someone from introducing a runtime error after following your advice without understanding the consequences.
  - WorldMaker 8y ago
    I just often find I need to remind people that there are other data types to consider reifying to beyond List/Array. While I'd probably suggest removing ToList() from Linq as List<T> really does seem to be the wrong data type more often than not, but for whatever reason so many people find it the "easiest" way to deal with Linq. I feel like ToLookup() and ToDictionary() should be more people's best friends when working with Linq. Too often I find bad searches and joins that become so much easier/more performant if some middle product was appropriately stored in a Dictionary<K, V> or ILookup<K, V>. Also with Linq against IQueryable sources, so often ToList/ToArray is used where AsEnumerable would be better. Understanding those monad boundaries is hard, and AsEnumerable doesn't always sound self-explanatory to people when they need to cross those boundaries. (I've thought before that some simple IDE highlighting of which bits of Linq are against IQueryable and which against IEnumerable might help some people think better in Linq.)
tluyben2 8y ago
People abuse Linq a lot though; enormously complex queries over very large datasets without really knowing what you are doing. When people need .Each, some will just do .ToList().Each(. Etc. I found a bigger issue even with abuse/overuse) (or use at all really) of dynamic. I wish there was a way of to ban it at compile time.
- jenscow 8y ago
  I will challenge every use of dynamic (and var, for that matter), unless it's used in the very few appropriate cases.
  - reubenbond 8y ago
    `var` is entirely syntactic sugar (compile-time type inference) and there's no runtime cost associated with it. I find it makes most (but not all) code more readable, particularly given that we have great IDEs/tooling in the .NET world.
    - ralphael 8y ago
      I agree, by using var you need to name your variable better therefore making your code more readable. Rather than relying on the interface/class definition to explain to someone why you used "obj".
      - quickthrower2 8y ago
        To var or not to var is purely a "tabs 'n' spaces" debate, but doesn't affect performance.
        jenscow 8y ago
        It will affect the performance of someone trying to read your code.
  - goto11 8y ago
reubenbond 8y ago
I feel I need to clarify this - my post is aimed specifically at optimizing code for high performance. Many of those tips will reduce readability of the code, which may make it harder to determine its correctness. I'm a big fan of LINQ and I use it in my own code, just not in the high-perf bits. LINQ is great for writing code which has "obviously no deficiencies" in terms of correctness. Without it your code may end up having "no obvious deficiencies". I also like LinqFaster, LinqAF and these other libraries/tools which can make LINQ usable in more domains.
deleted 8y ago
[deleted]
davidgrenier 8y ago
In that vein, Nessons has a Stream Fusion library: https://github.com/nessos/Streams https://github.com/nessos/Streams For lightweight threads I recommend https://github.com/Hopac https://github.com/Hopac which is an implementation of SML's John Reppy's Concurrent ML on .NET.

GordonS 8y ago

This looks really interesting - are the benchmarks in the GitHub repo against .NET Framework? With all the performance work that's been done in .NET Core, it might be interesting to compare it with .NET Core. Also, I noticed there haven't been any commits for a while - is this more or less considered 'complete'?

gameswithgo 8y ago
Yes it is more or less considered complete. I may do a pass over it to see if it can leverage or support span<t> better sometime soon. Performance deltas should be similar on .NET Core.

darrenkopp 8y ago

Also worth mentioning: https://github.com/kevin-montrose/LinqAF https://github.com/kevin-montrose/LinqAF Focused on reducing the allocations which makes linq heavy

WorldMaker 8y ago

So your library speeds up Array<T>/T[] and List<T> Linq operations? That's quite interesting, though usually in my experiences in Linq performance optimization the problem is usually prematurely using the wrong data type for your Linq operation. Given the immediate next bullet point in the article is about foreach operations over List<T>, that would be my top expectation why they were seeing so much allocation in Linq. ToList() allocates a lot, and to many projects I've seen reify everything with ToList() way too often in Linq usage. I've argued before that List<T> is very rarely the appropriate data structure for a lot of Linq work, and personally consider ToList() harmful. A successful strategy I've used to cleaning up Linq performance in projects is to simply start by remove all ToList() calls entirely and work to move API signatures to use smarter, more appropriate data types than List<T> and IList<T> everywhere.

caseymarquis 8y ago
You just need to be careful not to allow any subsequent modifications of the original collection when doing that (assuming you're using the returned LINQ result), or you'll introduce runtime errors. Typically, my use of ToList or ToArray means I'm guarding against future runtime errors due to collection modification at the expense of memory allocation. It sounds like you know this already, but I like to think I just saved someone from introducing a runtime error after following your advice without understanding the consequences.
- WorldMaker 8y ago
  I just often find I need to remind people that there are other data types to consider reifying to beyond List/Array. While I'd probably suggest removing ToList() from Linq as List<T> really does seem to be the wrong data type more often than not, but for whatever reason so many people find it the "easiest" way to deal with Linq. I feel like ToLookup() and ToDictionary() should be more people's best friends when working with Linq. Too often I find bad searches and joins that become so much easier/more performant if some middle product was appropriately stored in a Dictionary<K, V> or ILookup<K, V>. Also with Linq against IQueryable sources, so often ToList/ToArray is used where AsEnumerable would be better. Understanding those monad boundaries is hard, and AsEnumerable doesn't always sound self-explanatory to people when they need to cross those boundaries. (I've thought before that some simple IDE highlighting of which bits of Linq are against IQueryable and which against IEnumerable might help some people think better in Linq.)

tluyben2 8y ago

People abuse Linq a lot though; enormously complex queries over very large datasets without really knowing what you are doing. When people need .Each, some will just do .ToList().Each(. Etc. I found a bigger issue even with abuse/overuse) (or use at all really) of dynamic. I wish there was a way of to ban it at compile time.

jenscow 8y ago
I will challenge every use of dynamic (and var, for that matter), unless it's used in the very few appropriate cases.
- reubenbond 8y ago
  `var` is entirely syntactic sugar (compile-time type inference) and there's no runtime cost associated with it. I find it makes most (but not all) code more readable, particularly given that we have great IDEs/tooling in the .NET world.
  - ralphael 8y ago
    I agree, by using var you need to name your variable better therefore making your code more readable. Rather than relying on the interface/class definition to explain to someone why you used "obj".
    - quickthrower2 8y ago
      To var or not to var is purely a "tabs 'n' spaces" debate, but doesn't affect performance.
      - jenscow 8y ago
        It will affect the performance of someone trying to read your code.
- goto11 8y ago

reubenbond 8y ago

I feel I need to clarify this - my post is aimed specifically at optimizing code for high performance. Many of those tips will reduce readability of the code, which may make it harder to determine its correctness. I'm a big fan of LINQ and I use it in my own code, just not in the high-perf bits. LINQ is great for writing code which has "obviously no deficiencies" in terms of correctness. Without it your code may end up having "no obvious deficiencies". I also like LinqFaster, LinqAF and these other libraries/tools which can make LINQ usable in more domains.

deleted 8y ago

[deleted]

davidgrenier 8y ago

In that vein, Nessons has a Stream Fusion library: https://github.com/nessos/Streams https://github.com/nessos/Streams For lightweight threads I recommend https://github.com/Hopac https://github.com/Hopac which is an implementation of SML's John Reppy's Concurrent ML on .NET.