4 ms·
At least in my experience, problems with Linq are problems with optimization. For example, if you have an in-memory collection with 100,000 items, and you do co
by gwbas1c 7y ago
At least in my experience, problems with Linq are problems with optimization. For example, if you have an in-memory collection with 100,000 items, and you do collection.First(i => i.id == 123), that will be much slower than a dictionary indexed by id. (Refactor so you can do collection[123] instead of Linq.)
Some developers understand this, for other less savvy developers, it's just voodoo. In any case, if the collection is 100 items, there's no real advantage to optimizing.
Also, a foreach loop with an if statement is probably faster than a select statement, because it's not allocating a lambda and going back and forth in the stack; and a for loop indexed by a counter is probably faster than a foreach loop. Those, however, are micro-optimizations that really need their value shown in a profiler.
I once wrote a library that allowed indexing collections by different properties. It was quite intense, and it paid off as the application has lots of RAM-based queries and its internal collections can scale to 20,000 or 250,000 items.
(Unrelated: I wanted to open source the library, but my boss wouldn't let me. Sigh. Maybe I should have pushed back around my annual review that being able to put my name on a useful open-source library is important career development.)
- mathgladiator 7y agoSo, I'm building a language with linq and the solution to this is how data is represented and if the trade off between memory and cpu is expressible via indexing. It seems that linq in c# is simply syntactic sugar rather than deploying a static analysis to extract optimizations.
- WorldMaker 7y agoA problem is Linq in C# is both: Linq against IQueryable<T> is often a case where a query planner gets involved (more often than not an SQL Database's decades optimized Query Planner); IEnumerable<T> takes the simplest in-memory generator/iterable approach that makes sense (one like most functional programming languages have used for decades). I've thought for a long time that "coloring" IEnumerable<T> versus IQueryable<T> queries differently, at least at the syntax highlighting level, would go a long way to help people better their mental models of Linq. Especially at the complicated transition places where an IQueryable<T> query "falls back" to IEnumerable<T>.
- WorldMaker 7y agoIn my experience, so much of the problems with Linq are problems with knowing the right data structures for the job, and that's something of a universal problem, Linq just makes it easier to forget to pay attention to what your data structures are; the same code written equivalently in indexed for loops or foreach loops makes the exact same mistakes. More junior programmers are just at least somewhat more familiar at spotting where they are using the wrong data type in a for loop or foreach, partly because they'd recognize when they have more loops than they expected (.First() when alone is a loop). I've long joked that ToList() should be considered harmful, because it's an interesting crutch that gets over used and List<T> is so often the wrong data type for the operations to follow. Every time I have stepped in to performance optimize a Linq-heavy codebase, I've started by finding and removing nearly every use of ToList(). ToLookup() is one of the frequently unsung heroes of Linq that gets overlooked too often. It is the "indexing" operator, as ILookup<K, V> is the out-of-the-box multi-value Dictionary of .NET. I think ToLookup() gets overlooked often because there's no concrete, mutable Lookup<K, V> in the BCL (MultiValueDictionary<K, V> exists in the Experimental Collections NuGet package). Deleting unnecessary ToList()s (where a query was "cut in half", presumably for debug reasons) and then converting nearly every remaining instance of ToList() to ToDictionary() or ToLookup() based on queries/usage patterns that followed was often alone a huge performance boost to Linq code that I found, simply because it was using a more useful data structure. Sometimes you don't even have to do any of the more complicated performance testing/debugging work and can stop there at the easy stuff.