5 ms·

More info here - https://siderite.dev/blog/linq-in-javascript-linqer/ https://siderite.dev/blog/linq-in-javascript-linqer/ Pretty neat project. LINQ is a grea

by skrowl 7y ago

More info here - https://siderite.dev/blog/linq-in-javascript-linqer/ https://siderite.dev/blog/linq-in-javascript-linqer/ Pretty neat project. LINQ is a great time saver in .NET development, I'm looking forward to messing around with it in javascript.

gameswithgo 7y ago
its a shame that linq had the overhead it does when used with collections of objects. at work right now i am often having to de-linq things
- manigandham 7y ago
  Try LinqFaster which uses different techniques for better performance and less allocations: https://github.com/jackmott/LinqFaster https://github.com/jackmott/LinqFaster
  - gameswithgo 7y ago
    i wrote that :)
- sebazzz 7y ago
  Linq should not be used on hot paths, as it causes GC pressure. If this is what bothers your program, you can find out by profiling. However, if you have some slow SQL query or redundant SQL queries, I bet it is performance wise better to do something about that. In the case of Roslyn, the C# compiler, they avoid Linq on the hot paths (which run many times, for instance when typing in Visual Studio!) but outside hot paths linq is still used.
  - oaiey 7y ago
    Which is annoying as hell once you are used to LINQ. I completely forgot how to write a for loop till .NET Core came up with Span<T> and friendly forced us to re-think about memory. I hope they update LINQ in .NET to address these use cases.
- fuzzy2 7y ago
  I’m interested in what performance problems you’re seeing. Maybe then I won’t fall into the same trap in the future. Off the top of my head, the only possibility I see is allocating objects with `Select` or the like. If one can avoid that, the only overhead should be the predicate/selector method calls...?
  - gameswithgo 7y ago
    best way to get familiar with the degree of overhead is benchmark some things with benchmarkdotnet using linq and with for loops. lists and arrays and ienumerables. you will build up an intuition for the costs involved. groupby in particular is really nasty. any time you want to use that just put things into a dictionary instead.
    - fuzzy2 7y ago
      BenchmarkDotNet looks cool, thanks for that! The problem is that I never had problems! (With LINQ-to-Objects performance.) That’s why I’m asking in the first place. Maybe I intuitively avoided all the pitfalls, who knows. It would be cool if you could share some (abstract) operator chains that caused problems in whatever you’re working on.
      - gameswithgo 7y ago
        So I have some benchmarks in this repo showing some comparisons: https://github.com/jackmott/LinqFaster https://github.com/jackmott/LinqFaster While those can look really bad - orders of magnitude slower! Keep in mind with things like Where/Select, its a fixed amount of overhead. If you are larger amounts of work, or IO in those clauses, then the fixed amount of overhead is small in comparison. There are some common pitfalls you can avoid: Don't use GroupBy - it actually has a lot of overhead. Put things into a dictionary by hand instead. Instead of chaining together multiple Where clauses, just use one Where clause and the && operator. This saves you some overhead and has the same semantics. Consider whether you can presize a collection and then fill it up, rather than calling "ToList()" or "ToDictionary" because those will not presize the collection for you. The list will have to reallocate as it grows etc. Keep in mind that depending on your .NET version, calls to Count() may have to actually iterate over the whole collection even if that isn't necessary. .NET Core 3 fixes some of those cases, older versions may iterate over an IEnumerable backed by a List just to get the count, when the count is already there. At work right now, we just operate at very high scale, so in high traffic endpoints its worth it to pull microseconds of time out, and reduce memory use by bytes. Because everything gets multiplied by large numbers of requests.
hermitdev 7y ago
I got dinged pretty heavily for using Linq in an interview test at a hedge fund several years back. I was a little miffed because they jad issues with the memory allocations and speed. Thing is, this was an ahead of time test a3md there was no explanations about expectations beyond "do X" and "do Y", etc. Being a nonpaid test, I did what was expedient to me that met the requirements. Most of the questions were 1-liners using Linq, but that wasn't acceptable. I probably didn't make any friends when I pointed out that I solved all of their problems and that there were no other conditions on the solution. Fuck 'em. Interviewing with hedgefunds is a pain in the ass. Bunch of premadonnas that think their shit doesnt stink (but is probably the rankest WTF code you'll ever see) - and I say this having spent a decade at one. A separate fund gave me a private hackerrank challenge. My code passed all of the public requirements and examples, but failed a private test case. No info given about the failure. Emailed the recruiter about my frustration with it, said fuck it, I'm not interested. Figured if theyre like that in an interview, theyre probably colossal pricks to actually work with. No surprise, I didnt get an on site interview.
knocte 7y ago
Why not just use C# instead of staying in the shitty alternative? You know... https://twitter.com/knocte/status/1207544365607985152 https://twitter.com/knocte/status/1207544365607985152
- mythz 7y ago
  Both these expressions return the same result in both C# and JavaScript: 0.1 + 0.1 == 0.2; //= true 0.1 + 0.2 == 0.3; //= false It's not the language, it's https://floating-point-gui.de https://floating-point-gui.de
  - knocte 7y ago
    I was going to reply that I thought the point of the video/tweet is that JS does choose floating point by default while others don't. But I stand corrected, C# default sucks too: it uses the double type! And here I thought that by using F# I would even be protected from this shit but unfortunately it has the same problem... Performance should be opt-in! (favouring correctness over it) :facepalm: Thanks
    - wruza 7y ago
      First you should state the correctness level you need. 0.2? 1/7? sqrt(2)+pi? int n=0, n<inf, sum(1/n^3)?
    - sasmithjr 7y ago
      Look at the decimal type; it's less performant, but you'll avoid floating point issues. https://docs.microsoft.com/en-us/dotnet/api/system.decimal?view=netcore-3.1 https://docs.microsoft.com/en-us/dotnet/api/system.decimal?v...
      - ygra 7y ago
        decimal is still floating-point, just decimal floating-point instead of binary. Things like 1/3 can still not be represented exactly. It's just that for values given in base 10 there's no transformation of the representation and 0.3 remains exactly 0.3.

by skrowl 7y ago

gameswithgo 7y ago

its a shame that linq had the overhead it does when used with collections of objects. at work right now i am often having to de-linq things

manigandham 7y ago
Try LinqFaster which uses different techniques for better performance and less allocations: https://github.com/jackmott/LinqFaster https://github.com/jackmott/LinqFaster
- gameswithgo 7y ago
  i wrote that :)
sebazzz 7y ago
Linq should not be used on hot paths, as it causes GC pressure. If this is what bothers your program, you can find out by profiling. However, if you have some slow SQL query or redundant SQL queries, I bet it is performance wise better to do something about that. In the case of Roslyn, the C# compiler, they avoid Linq on the hot paths (which run many times, for instance when typing in Visual Studio!) but outside hot paths linq is still used.
- oaiey 7y ago
  Which is annoying as hell once you are used to LINQ. I completely forgot how to write a for loop till .NET Core came up with Span<T> and friendly forced us to re-think about memory. I hope they update LINQ in .NET to address these use cases.
fuzzy2 7y ago
I’m interested in what performance problems you’re seeing. Maybe then I won’t fall into the same trap in the future. Off the top of my head, the only possibility I see is allocating objects with `Select` or the like. If one can avoid that, the only overhead should be the predicate/selector method calls...?
- gameswithgo 7y ago
  best way to get familiar with the degree of overhead is benchmark some things with benchmarkdotnet using linq and with for loops. lists and arrays and ienumerables. you will build up an intuition for the costs involved. groupby in particular is really nasty. any time you want to use that just put things into a dictionary instead.
  - fuzzy2 7y ago
    BenchmarkDotNet looks cool, thanks for that! The problem is that I never had problems! (With LINQ-to-Objects performance.) That’s why I’m asking in the first place. Maybe I intuitively avoided all the pitfalls, who knows. It would be cool if you could share some (abstract) operator chains that caused problems in whatever you’re working on.
    - gameswithgo 7y ago
      So I have some benchmarks in this repo showing some comparisons: https://github.com/jackmott/LinqFaster https://github.com/jackmott/LinqFaster While those can look really bad - orders of magnitude slower! Keep in mind with things like Where/Select, its a fixed amount of overhead. If you are larger amounts of work, or IO in those clauses, then the fixed amount of overhead is small in comparison. There are some common pitfalls you can avoid: Don't use GroupBy - it actually has a lot of overhead. Put things into a dictionary by hand instead. Instead of chaining together multiple Where clauses, just use one Where clause and the && operator. This saves you some overhead and has the same semantics. Consider whether you can presize a collection and then fill it up, rather than calling "ToList()" or "ToDictionary" because those will not presize the collection for you. The list will have to reallocate as it grows etc. Keep in mind that depending on your .NET version, calls to Count() may have to actually iterate over the whole collection even if that isn't necessary. .NET Core 3 fixes some of those cases, older versions may iterate over an IEnumerable backed by a List just to get the count, when the count is already there. At work right now, we just operate at very high scale, so in high traffic endpoints its worth it to pull microseconds of time out, and reduce memory use by bytes. Because everything gets multiplied by large numbers of requests.

hermitdev 7y ago

I got dinged pretty heavily for using Linq in an interview test at a hedge fund several years back. I was a little miffed because they jad issues with the memory allocations and speed. Thing is, this was an ahead of time test a3md there was no explanations about expectations beyond "do X" and "do Y", etc. Being a nonpaid test, I did what was expedient to me that met the requirements. Most of the questions were 1-liners using Linq, but that wasn't acceptable. I probably didn't make any friends when I pointed out that I solved all of their problems and that there were no other conditions on the solution. Fuck 'em. Interviewing with hedgefunds is a pain in the ass. Bunch of premadonnas that think their shit doesnt stink (but is probably the rankest WTF code you'll ever see) - and I say this having spent a decade at one. A separate fund gave me a private hackerrank challenge. My code passed all of the public requirements and examples, but failed a private test case. No info given about the failure. Emailed the recruiter about my frustration with it, said fuck it, I'm not interested. Figured if theyre like that in an interview, theyre probably colossal pricks to actually work with. No surprise, I didnt get an on site interview.

knocte 7y ago

Why not just use C# instead of staying in the shitty alternative? You know... https://twitter.com/knocte/status/1207544365607985152 https://twitter.com/knocte/status/1207544365607985152

mythz 7y ago
Both these expressions return the same result in both C# and JavaScript: 0.1 + 0.1 == 0.2; //= true 0.1 + 0.2 == 0.3; //= false It's not the language, it's https://floating-point-gui.de https://floating-point-gui.de
- knocte 7y ago
  I was going to reply that I thought the point of the video/tweet is that JS does choose floating point by default while others don't. But I stand corrected, C# default sucks too: it uses the double type! And here I thought that by using F# I would even be protected from this shit but unfortunately it has the same problem... Performance should be opt-in! (favouring correctness over it) :facepalm: Thanks
  - wruza 7y ago
    First you should state the correctness level you need. 0.2? 1/7? sqrt(2)+pi? int n=0, n<inf, sum(1/n^3)?
  - sasmithjr 7y ago
    Look at the decimal type; it's less performant, but you'll avoid floating point issues. https://docs.microsoft.com/en-us/dotnet/api/system.decimal?view=netcore-3.1 https://docs.microsoft.com/en-us/dotnet/api/system.decimal?v...
    - ygra 7y ago
      decimal is still floating-point, just decimal floating-point instead of binary. Things like 1/3 can still not be represented exactly. It's just that for values given in base 10 there's no transformation of the representation and 0.3 remains exactly 0.3.