12 ms·

Net 9.0 LINQ Performance Improvements

203 points by olvy0 2y ago

high_na_euv 2y ago
LINQ is so fucking useful and well designed feature of .NET ecosystem that it is unreal when you gotta use lang which doesnt have such a thing. C# design team is/was unparalleled
- pjmlp 2y ago
  LINQ is largely based on FP stuff, also how Smalltalk collections work. It is relatively easy to find similar capabilities in most languages nowadays, unless one is stuck on Go, C and similar.
  - sanex 2y ago
    Do you know an equivalent for Linw to EF in kotlin or Java because I have not found it.
    - stanac 2y ago
      Those are linq expressions. They are indeed wonderful. You get an abstract tree from which you can create SQL or API commands to access the data source. I remember in the early days (.NET 3.5?) there were multiple examples of LINQ2X like Linq2Csv, Linq2Rss Linq2Drobox (I'm paraphrasing, I don't remember actual examples, but it was wild). There is also relinq library which transforms linq expressions into expressions which are easier to understand/use.
    - pjmlp 2y ago
      Rather quick search, as I am more of a myBatis person, Java: https://www.jooq.org/ https://www.jooq.org/ Kotlin: https://www.ktorm.org https://www.ktorm.org
  - kumarvvr 2y ago
    Any pointers to such libraries in python?
    - pjmlp 2y ago
      itertools would be the starting point, unfortunelly Python is rather limited due to the way it only supports one line lambdas.
  - apwell23 2y ago
    No it isn't easy to find similar capabitites in java, go, python, ruby. Maybe you do simulate some of this using meta programming in ruby but its certainly not 'easy to find'.
    - pjmlp 2y ago
      It certainly is, unless you are talking about SQL like syntax, which is basically syntax sugar for classical FP. And I explicitly left Go out of my list.
      - apwell23 2y ago
        >unless you are talking about SQL like syntax yes thats what linq is? https://learn.microsoft.com/en-us/dotnet/csharp/linq/ https://learn.microsoft.com/en-us/dotnet/csharp/linq/ "Language-Integrated Query (LINQ) is the name for a set of technologies based on the integration of query capabilities directly into the C# language." With LINQ, a query is a first-class language construct, just like classes, methods, and events. doing this in java is not LINQ imo List<Integer> lowNums = filter(toList(numbers), new Predicate<Integer>() { @Override public boolean apply(Integer n) { return n < 5; } });
        high_na_euv 2y ago
        To be fair, 99% of linq usage is method chaining syntax, not query syntax
        apwell23 2y ago
        LINQ = Language Integrated Query Its even in the name. What do you mean by "method chaining is linq" ?
        ygra 2y ago
        Considering the methods live in the System.Linq namespace, I think the extension methods may also be called LINQ.
  - blackoil 2y ago
    One difference with LINQ is its ubiquity. It works with database, in memory data structures, on disk files. You can use your skills/code across all the system.
    - John23832 2y ago
      It's just built on top of anything that is Iterable. If a language has first class iterator support, they could do something similar.
      - naasking 2y ago
        Mainly first-class functions I think. If you have those, you can just use fold in the core combinators.
      - mythz 2y ago
        Takes a lot more than that, LINQ providers work by accepting a LINQ Expression Syntax tree instead of an opaque function, which allows providers to inspect and traverse the Expression's AST and translate it into the data source it's implementing. This Expression AST is constructed by the compiler, not something that can be tacked on by a library later.
        megadal 2y ago
        Yes, but I think the point is practically every high level language can already do this pretty trivially. If it's scripted you can typically just get a string representation of the function. If it's Java, JAR inspection/dynamics have been a thing for a long time. And in other languages, they usually directly support metaprogramming (like Rust) and plugging code into the compilation logic.
        mythz 2y ago
        If it were trivial you'd see LINQ-like providers implemented in "practically every high level language". Source code of the function means you have to implement the parser/lexer to convert it into a usable AST which is bad for both runtime performance and library size. Very much doubt this is available in Java, which Java ORM lets you use native Java language expression syntax to query a database?
  - jiehong 2y ago
    I've found [0] for clojure, which maps the best IMO, but it also contains links to the same LINQ examples in other languages (java, kotlin, swift, elixir, python, ...). [0]: https://github.com/mythz/clojure-linq-examples https://github.com/mythz/clojure-linq-examples
    - nightski 2y ago
      The difference is many of those are dynamically typed languages. It's still useful, but a lot of the a beauty of LINQ comes from the fact that it is within a statically typed language.
  - John23832 2y ago
    It's surprising that Go didn't ship with it, but given that they just added iterators, it's coming. Rust has combinators, which is the same thing. Most new languages are recognizing that functional support (even if they don't consider themselves FP languages) is necessary.
    - rw_panic0_0 2y ago
      nah it's not coming, functions like maps and filters won't come to go by design, iterators are not only about FP stuff
    - pjmlp 2y ago
      Go culture is quite clearly against this kind of niceties.
    - devjab 2y ago
      You can do functionality similar to LINQ with chaining as long as you don’t need to call a method with a generic different from the one defined. If you do need that, you’re going to have to do it without chaining. You can still do something similar but it’ll be a lot less elegant than how C# does it. It’s part of the design philosophy of Go though. They don’t want any magic. It’s similar to why they enforce explicit error handling instead of allowing you to chose between explicit and implicit. They want you to write everything near where it happens and not rely on things you can’t see. It’s probably the primary reason that Go is either hated or loved. I think it’s philosophy is great, a lot of people don’t. I have written a lot of C# over the years, so I’m a little atypical in that regard, I think most C# developers think Go is fairly inferior and in many regards they are correct. Just not in the ones that matter (come at me!). To elaborate a little on that, Go protects developers from themselves. C# is awesome when it’s written by people who know how it works, when it’s not you’ll get LINQ that runs in memory when it really shouldn’t and so on.
  - whizzter 2y ago
    Yes and no, the LINQ syntax being coherent between IEnumerable<> and IQueryable<> hides a lot of good magic. IEnumerable<> is regular in-memory lambdas/streams, same what you find in many places. IQueryable<> relies on the LINQ expressions, those CAN be JIT compiled for direct execution, but the fact that they are data-objects is what allows the translation to SQL and execution on the server rather than locally and can give massive gains since processing can be done where the data lives.
    - whizzter 2y ago
      For reference, to achieve what IQueryable does with 100% normal code in JavaScript you need something like Qustar that was posted here a month ago. Regular transform code in JS (Like IEnumerable) const ids = users.filter(user => user.age<18).map(user => user.id); IQueryable like to be transformed to the server: const ids = users.filter(user => user.age.lt(18)).map(user => user.id); In C# it'd look identical, but in JS or Java this would be achieved via proxy-object hacks (the .lt() function in the filter instead of the < operator and the .id property getter for the user map to send a flag under the hood for the mapping function). https://github.com/tilyupo/qustar https://github.com/tilyupo/qustar
- munchler 2y ago
  If you like LINQ, you should really give F# a try.
  - neonsunset 2y ago
    F# is even better. I only wish performance wasn't hit or miss around sequences/iterator expressions. Hopefully one day it will reach parity with C# through compiler improvements and/or detaching FSharp.Core from conservatively targeting NS2.0 and modernizing it.
    - emn13 2y ago
      I never really got over the order-dependent and fairly slow compile times, but it's been like 10 years since I used it for anything even slightly complex. Is F# better in this regard now, or are there accessible patterns to help deal with that?
      - munchler 2y ago
        Order-dependency is a feature, not a bug, but it does take some getting used to. If you really want to avoid it, you can now declare all functions in a module to be mutually recursive, so you can put them in any order. Cyclic dependencies between files are still prohibited, for good reason. See https://fsharpforfunandprofit.com/posts/cyclic-dependencies/ https://fsharpforfunandprofit.com/posts/cyclic-dependencies/. The F# compiler is slower than the C# compiler, but it's still more than fast enough for building large applications.
      - throwaway31424 2y ago
        [dead]
- mhh__ 2y ago
  Maybe you've been unlucky but LINQ didn't really seem all that interesting to me using it. It's pretty well put together but it was very hard to work out the patterns of what it was doing underneath e.g. I could not tell you now how to implement a custom IQueryable (I know where to look but couldn't tell you the rhythms of it) for some database and I am the type of person who usually does go and find that kind of thing out before using something for a "serious" project. Maybe it's just a microcosm for C# as a whole - very productive language, good design team, quite hobbled by it's weird/philistine upbringings: Bjarne said within C++ there is a simple language trying to escape, in C# you basically have a simple language buried in nouns.
  - pjc50 2y ago
    > It's pretty well put together but it was very hard to work out the patterns of what it was doing underneath e.g. I could not tell you now how to implement a custom IQueryable There's a lot hidden in there, but basically they expect you to use EF. Writing an IQueryable is a similar amount of work to writing a SQL query planner. You get passed a tree of Expression objects. https://learn.microsoft.com/en-us/archive/blogs/mattwar/linq-building-an-iqueryable-provider-part-i https://learn.microsoft.com/en-us/archive/blogs/mattwar/linq...
    - SideburnsOfDoom 2y ago
      > basically they expect you to use EF. Writing an IQueryable... I don't agree. I don't feel any expectation to use EF. It would not be relevant anyway to our code. LINQ is not EF and EF is not LINQ. EF uses LINQ but not vice versa. LINQ is useful without EF. The LINQ extension methods that we use constantly are on IEnumerable<T> so EF and IQueryable is of no importance to us, but LINQ is used everywhere.
      - pjc50 2y ago
        > IQueryable is of no importance to us I was discussing IQueryable with someone else to whom it was important. In reply to > I could not tell you now how to implement a custom IQueryable (I know where to look but couldn't tell you the rhythms of it) for some database And "for some database" the default answer is "use EF" as the intermediary between LINQ queries and the database itself, rather than delving into IQueryable. LINQ-to-objects is very important and useful but I was talking about something else.
- hggigg 2y ago
  LINQ is a veritable footgun in any large team I find. While it's extremely powerful and really nice, it's so so so easy to blow your toes off if you don't know what you are doing. Some of my favourite screw ups I saw: * Not understanding when something is evaluated. * Not understanding the computational complexity of multiple chained operations/aggregates. * Not understanding the expectation that Single() requires exactly one of something. * Not understanding how damn hard it is to test LINQ stuff.
  - high_na_euv 2y ago
    >Not understanding the expectation that Single() requires exactly one of something. Sorry, but idk how it is footgun of LINQ. It is like complaining about 0 or 1 based indexing >Not understanding how damn hard it is to test LINQ stuff. Any examples? Because I struggle to see such
  - pjc50 2y ago
    How much of that is LINQ-specific and how much is just the cost of using an ORM to build queries rather than typing them out as SQL? I've never encountered testing problems with LINQ-to-objects.
  - John23832 2y ago
    I sort of agree. I recently had to code splunk a bug with 3 other engineers and we all got to a relatively complex LINQ and of the 4 of us, we all had 4 different interpretations when visually inspecting. > Not understanding how damn hard it is to test LINQ stuff. I disagree with this. Just run the LINQ query on a compatible iterable.
    - hggigg 2y ago
      Regarding the testing, it's more the sheer multiplicative number of cases you have to consider on a LINQ expression.
    - SideburnsOfDoom 2y ago
      > I recently had to code splunk a bug with 3 other engineers and we all got to a relatively complex LINQ and of the 4 of us, we all had 4 different interpretations when visually inspecting. You can write hard to read code with any framework. Yes it takes effort sometimes to make linq code clear, but you should not give up on it.
- highwaylights 2y ago
  I’ll get started then end up on a rant but.. This is really the thing with the entire .NET stack that’s very hard to communicate. The standard library and framework design are so well thought out relative to anything else out there. More than that, the support within VS is beyond any other dev tool that exists for any other language - it’s not even particularly close. Edit-and-continue comes to mind, which despite how many times people confuse the two is not hot reload, and is wildly more productive and useful. I remember back close to 20 years ago DHH was espousing Ruby/Rails and that the concept of types at all were a code smell, and thinking “you’re just very wrong, and clearly aren’t familiar with what else is out there”. Eventually a lot of that crowd moved to Node, then to typescript, and came around. VS Enterprise (expensive as it is) had features 15 years ago that still seem magical when I show them to JS/TS folks now. IntelliTrace is one that comes to mind - there’s nothing remotely close to it’s snapshot debugging that I’ve seen anywhere else, and I’ve really looked. The big problems with the ecosystem are that the docs are exhaustive but terribly boring, and not well explained from a learning-something-for-the-first-time perspective. They also really expect that everything you do is the Microsoft way, so if you’re trying to interface your code with something like an Avalonia UI, you’re on your own. The language is absolutely wonderful though, even when used with Rider. The productivity relative to node/typescript is better enough that it crushes my soul having to go back to wrestling tsconfig and imports after working with .NET references for even small changes. So many of the little things I used to take for granted really just work, and work well. It’s just a wonderful piece of work executed over decades, held back by poor community outreach and badly written documentation.
  - high_na_euv 2y ago
    Your experiences are coherent with mine. Developer experience is far ahead any other technology out there. Std lib and its API design is world class, I wish cpp had as good stdlib. Tooling is strong, especially debugger
  - bob1029 2y ago
    Agree with all of this. I think the biggest problem with popularity around these tools is that they're too good. When everything just works, you have a lot more time to focus on your problem. Once you can focus on your problem, you may quickly find out you don't actually care about it. Focusing on tools is a great way to hide from this reality.
- megadal 2y ago
  It's just an API for JIT, basically metaprogramming. It's cool but you can definitely do a similar thing in pretty much every high level language. With scripting languages, it's all JIT :) The C# teams progress on this has been slow. Keep in mind the CIL bytecode has had such capabilities for at least 20 years now and only in the past like decade are we seeing more features and optimizations around LINQ and System.Reflection.Emit. Dynamics were extremely slow in C# and if you look at the CIL generated you see why. It's possible for example to use something like a Haxe anonymous types[1] to optimize Dynamics so that CallSite caching is way more performant. I am pretty sure in C# the only way to accept an anonymous type is as a dynamic value, so even though the type of the structure is well-defined at compile-time, it will still rely heavily on runtime reflection/DLR with no additional caching beyond what DLR does for any other dynamic type. Anyways, this leads to niche libraries being built for handling dynamic data like JSON performantly. Which leads to annoying things like .NET libraries/apps being incompatible (without some adapter) if they use for example, different JSON libraries under the hood. (See [2]). Problems like these (the lack of actually good JIT/dynamic code support) in my opinion significantly slow down the .NET ecosystems development, that's why it always feels like .NET is just catching up with features other popular languages have. To be fair though, much of C#'s lag is owed to Microsoft's contribution to .NET being mostly technical debt. Almost everything good that came out of .NET came from open source/non MS teams (like Mono). [1] - https://haxe.org/manual/types-anonymous-structure.html https://haxe.org/manual/types-anonymous-structure.html [2] - https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/how-to https://learn.microsoft.com/en-us/dotnet/standard/serializat...
  - jeswin 2y ago
    > It's cool but you can definitely do a similar thing in pretty much every high level language. No. When it was release (circa 2007), very few mainstream languages embraced "Code as Data" the way C# did. In Java, there was no way to pass an expression (as an AST) to an SQL library. Which is why LINQ is so much more ergonomic than Hibernate. In C#, you could use language features you're already familiar with (such as "order.id > 100 && order.id < 200") in your queries, whereas Hibernate made you learn the framework's specific patterns (add Criteria etc etc, I don't recall now). Java just wasn't expressive enough for this. In fact, you couldn't do this even today in a language like say Python or JS. I mean, not without running it through something like babel to get an AST, and having arbitrary rules on what's code and what's data. C# had this in the spec; based on whether it was IQueryable. > Almost everything good that came out of .NET came from open source/non MS teams (like Mono). My team adopted Mono very early - like in 2005. Your statement is not true at all. C# and the Framework was a very good spec irrespective of what Open Source / Mono did, and while Mono existing might have accelerated .Net's transition into Open Source, it would have happened anyway due to the definitive swing towards Open Source in the 2000s. Linq-to-SQL, Asp.Net MVC, EF etc didn't come out of Mono.
- deleted 2y ago
  [deleted]
neonsunset 2y ago
Relevant section in the annual book-sized post on all performance improvements in .NET 9: https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-9/#linq https://devblogs.microsoft.com/dotnet/performance-improvemen... (for some reason HN would not allow to resubmit it, and the post was left by wayside not reaching frontpage)
dingdingdang 2y ago
OK, so since I've never actually bothered to look at what LINQ actually is.. there may be others in the same boat: "LINQ allows you to query any enumerable collections such as List<T>, Array, or Dictionary<TKey,TValue>.". Got to admit I still find the LINQ to DB passtru rather opaque even after reading up on it: https://stackoverflow.com/questions/30716776/passing-a-query-directly-through-linq-to-the-db/30716910#30716910 https://stackoverflow.com/questions/30716776/passing-a-query... Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL (in my optics anyway). Does anything library wise come close to this within the Golang ecosystem?
- caseymarquis 2y ago
  While LINQ does include a library of extension methods for functional programming with .NET collections (which is great), it also includes "Expression Classes". In a nutshell, this allows a user to pass a single expression lambda to a function, and the function implementor receives the abstract syntax tree for the lambda, not the lambda itself. You can then not only receive and analyze these trees, you can also manually build and compile them. This effectively allows a limited set of runtime macros within .NET.
- pjc50 2y ago
  > Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL. Pretty much. There's the "language integrated" version which looks a lot like SQL: var foo = new List<int> { 1, 2, 3, }; var response = from x in foo where x > 1 select x.ToString(); But that just translates to the method-orientated one which many people prefer response = foo.Where(x => x > 1).Select(x => x.ToString()); If instead of querying a List or Dictionary you query a database, using an ORM (usually Entity Framework), that will actually get converted to SQL and run on the DB.
- naasking 2y ago
  > Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL (in my optics anyway) It's more general and reusable than SQL, so you can map a subset of it to SQL, which is what some object-relational mappers do.
blackoil 2y ago
Anyone knows of a comprehensive book/tutorial to learn end to end web development in dotnet. Most I have found are elementary, outdated or poor quality.
- DeathArrow 2y ago
  I've learned by doing, but here you go. Books: C# 12 and .NET 8 - Modern Cross-Platform Development Fundamentals - Eighth Edition: Start building websites and services with ASP.NET Core 8, Blazor, and EF Core 8 by Mark J Price Web API Development with ASP.NET Core 8: Learn techniques, patterns, and tools for building high-performance, robust, and scalable web APIs by Xiaodi Yan Turorials: series of tutorials on YouTube by IAmTimCorey and Shawn Wildermuth.
- littlecranky67 2y ago
  That is because all the hot-and-new in .NET web development is Blazor, and it is not really popular outside of Microsoft's Blogosphere (and IMHO never will be and go down the same way as Silverlight did). The "venerable" technologies are still present in .NET 9 and still work, get maintained etc. Doing web dev in .NET nowadays for me is mostly creating HTTP/JSON/REST APIs and using whatever FE framework you like to interface it (for me that is React or NextJS). The keyword you want to search for is "ASP.NET WebApi" or - more modern - "ASP.NET minimal API". You could still do .NET MVC server-side rendering using "Razor" (the markup language in ASP.NET MVC - search for "ASP.NET MVC Razor".
  - WorldMaker 2y ago
    Blazor for better and worse can't have the same ending as Silverlight did. In Server Mode it is just Razor + SignalR for HTMX-style frontends that sort of live update/"magic RPC". In Client Mode it is just Razor as a virtual DOM-ish framework running directly in Web Assembly (wasm) in the Browser. At the end of the day Client Blazor is much more like writing front end websites with Elm than writing websites for the Silverlight plugin. (Hybrid Blazor where you use both server and client possibly in the exact same assembly/project is where most of the worst Blazor mistakes lie, but those aren't the mistakes of Silverlight, those are the mistakes of ASP Classic before .NET where you can be easily confused by new versions of the which bit of this is `runat="server"` and which bit is `runat="client"` and easily and accidentally create wild RPC ravioli/spaghetti code.)
DeathArrow 2y ago
The more C# borrows from F#, the happier I am. I am awaiting for discriminated unions to finally reach C# so I can do domain modelling like a boss. :)
- CharlieDigital 2y ago
  You can largely already pretty easily incorporate DUs using OneOf[0] and Dunet[1]. A practical example of using this: https://chrlschn.dev/blog/2024/07/csharp-discriminated-unions-and-dotnet-channels/ https://chrlschn.dev/blog/2024/07/csharp-discriminated-union... [0] https://github.com/mcintyre321/OneOf https://github.com/mcintyre321/OneOf [1] https://github.com/domn1995/dunet https://github.com/domn1995/dunet
- stult 2y ago
  I'd kill for units of measure too. It makes maintaining any kind of engineering or scientific code so much easier.
  - fuzzy2 2y ago
    In a past project, we used Units.NET. It worked pretty well as far as I could tell. (I was not involved with the gritty domain details.) My colleagues even extended the library where units were missing.
  - Jordanpomeroy 2y ago
    In my experience units of measure in technical apps is only interesting in the UI layer as most apps I’ve worked on have a “base” system of units that all calcs are expecting. What alternate experience have you seen?
    - greggyb 2y ago
      Units of measure in F# are useful for more than unit conversion workloads. They are compile-time type information with no runtime overhead. So if you're using a lot of interrelated arrays, you can make a unit of measure for indices into just a specific type of array. This allows type-level enforcement of never using the wrong index for the wrong array. Anywhere you might want to keep categories (not category theory) of numeric values distinct from one another without intermixing, units of measure in F# can cover you.
olvy0 2y ago
For me, the most useful part of LINQ is neither the IQueryable syntax tree based extension mechanism, nor the language integrated part (which I dislike), but simply the IEnumerable extensions. Originally known somewhat confusingly as linq to objects. Those allow me to write c# in functional style, keeping the code concise. The post I submitted refers mostly to optimizations to those extension methods. This clicked for me after having learned Haskell. It also shares some of Haskell's features/pitfalls, such as laziness. There are pitfalls, sure, and honestly I wouldn't advise a team having no one somewhat experienced with basic functional idioms (including laziness) to use it. It can lead to obtuse and slow code if used indiscriminately. I try to lead by example myself.
- osigurdson 2y ago
  C# has quite a few easy to use things that take a while to understand. In some ways it is a bit of an "experts only" language.
  - aksss 2y ago
    It’s hard to forget what you know and see through eyes of a beginner, but I definitely find C# to be a very “rich” language, in the sense of finding new features and ways of doing things the more I use it and as the incessant releases roll out. I like to think it’s still pretty easy to start with though if one is focused on the fundamentals. Lots of head room for progressively expert use, I guess.
- jasonthorsness 2y ago
  This is how I use LINQ as well. With some non-standard names, it has everything you need! Eric Lippert wrote a great series on monads tying it to LINQ: https://ericlippert.com/2013/04/02/monads-part-twelve/ https://ericlippert.com/2013/04/02/monads-part-twelve/
  - karmakurtisaani 2y ago
    Why on earth they named map as s Select and flatmap as SelectMany, I'll never understand. Still vert useful tho. (I first learned FP with Scala, so the names from there feel the most natural to me, tbh) Edit: it just occurred to me it has to be SQL inspired or something like that.
nightski 2y ago
Whenever I work in other languages/ecosystems LINQ is the one thing that I really miss. It's just such a nice capability to have available in the standard library. Beautifully designed given the constraints it had to work within.
EVa5I7bHFq9mnYK 2y ago
It's a shame, actually, that .NET performance improvements of up to x1000 could still be found after two decades and hundreds of millions spent on development.
- deleted 2y ago
  [deleted]
- eknkc 2y ago
  Most of the time, it is not because there were too many slow things to be improved, it is mostly because they are adding more facilities to the runtime, enabling other performance improvements. For example, the ReadOnlySpan type is a recent addition to the runtime and it will allow faster iterations when used. They simply enabled Linq to use it now.
  - EVa5I7bHFq9mnYK 2y ago
    ReadOnlySpan is a breakthrough innovative data structure, consisting of a pointer and a _length, that took Microsoft Corporation two decades to invent. Well, better late than never.
    - neonsunset 2y ago
      other languages do not have special pointers that can point to GC object interiors, be transparently addressed with arithmetics and cooperatively updated by GC without pinning, while also allowing to point to stack and unmanaged memory
      - EVa5I7bHFq9mnYK 2y ago
        I see, thanks. Still, x1000 performance improvement is called a bug fix.
        neonsunset 2y ago
        I encourage you to read through this section: https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-9/#linq https://devblogs.microsoft.com/dotnet/performance-improvemen... and go through the pull requests that it references. The Nx elements figure is a speedup you would expect if you introduce a shortcut to any algorithm that can bypass doing full iteration, and >100x speed-up is what you would expect from going from interface dispatch per element (even if devirtualized behind a guard) to a bespoke vectorized implementation. Even if you happen to have irrational dislike of .NET, this is still useful learning material and perfectly applies to other languages that expose similar functionality (provided they have sufficiently good type system and generics).
issafram 2y ago
[flagged]
- deleted 2y ago
  [deleted]
ibejoeb 2y ago
> Some more optimizations can happen when the chain ends up with methods like Count(), First(), Last(), ElementAt() or Sum(). For instance, OrderBy(criteria).First() can be optimized to execute as Min(criteria). As useful as that might be, we should really be writing better code. This is interesting for dynamically generated chains, I suppose. But if these operations are done on bespoke code, this is kind of perverse positive reinforcement. The library itself is recognizing a sub-optimal pattern and correcting for it. I hope there's feedback, at least, that suggests improvements to the underlying code.
hakanderyal 2y ago
Beware, once you get used to using LINQ and usually work in a domain that it shines, you won't want to go back to using anything else.
- jayd16 2y ago
  Do not, my friends, become addicted to LINQ. It will take hold of you, and you will resent its absence.
- klysm 2y ago
  It’s still less powerful than something like polars
stonethrowaway 2y ago
C# is Pragmatic [0]. [0] https://news.ycombinator.com/item?id=41761346 https://news.ycombinator.com/item?id=41761346
mannycalavera42 2y ago
I have fond memories of using LINQPad That's why I guess today I work in Clojure :)
- m_fayer 2y ago
  It’s been consistently one of my favorite tools for a good decade. I don’t know how much Microsoft is paying them to keep it Windows-only because that’s the only reason I keep windows around, and I’m sure that goes for a number of important developers.
  - stuartd 2y ago
    There’s a Mac version coming (very exciting!) https://x.com/linqpad/status/1820379112651026587 https://x.com/linqpad/status/1820379112651026587
  - phito 2y ago
    I'm using NetPad on Linux. Still far away from LinqPad but it's pretty good.
- iLemming 2y ago
  I can relate. I used .Net at the beginning of my career, and that one was one of the few nice things. I enjoyed building in C#, until one day I realized that MSFT is more like a mental prison (apologies for the strong negative word) - mind that it's my personal feeling at that time, many things have changed positively since then, although I moved on already. Also, I too, use Clojure today - it is very nice.
louthy 2y ago
"LINQ Performance improvements" really should read: "our own List<T> implementation performance improvements". Microsoft seem to spend their time improving what they need, rather than spending time on general improvements. LINQ (the syntax, not the method extensions) needs investment. Primarily around lambda allocation and potentially on compile-time reduction of lambdas. It's time to have value-type local lambdas, or a strategy that doesn't make lambda allocation the overhead that it is. And also there really should be wildcard support (`_`) for LINQ variables by now. Which was been completely ignored when they were brought in for lambdas. It should also be possible to use a lifted-type (like IEnumerable<T>, Option<T>, etc.) as the final item in a LINQ expression, rather than `select ...`. The select adds overhead that isn't needed for certain use-cases and limits things like tail-recursive LINQ expressions. Libraries like mine that go all-in on LINQ [1], but aren't using `IEnumerable` or `IQueryable`, or any of the LINQ extensions, continually get ignored because MS are focusing purely on improving the performance of their own projects. A good example is the improved lambda inference. It was only brought forward because ASP.NET Core needed it for its minimal API. It seems like many of the features of the language/framework are driven by their own needs rather than those of the community. The absolute worst thing is the ever expanding set of 'magic methods' (like the LINQ extensions Select, SelectMany, and Where, but also GetAwaiter, and the rest). MS are adding capabilities for themselves (the compiler mostly) rather than do what is really needed and add proper higher-kinded traits to resolve the magic. So everything is weakly typed and only vaguely discoverable by the compiler :/ LINQ is one of those key differentiators between languages, yet it's been wallowing, pretty much untouched since C# 3. I think it's a crying shame that it has basically been ignored since then and, even now, they think LINQ is only useful for iterating lists. And primarily only their own list implementations. /rant Don't get me wrong, I appreciate all performance improvements, I'm sure it will help plenty of users. But the focus always seems to be narrowly targeted, which limits the potential. [1] https://github.com/louthy/language-ext/ https://github.com/louthy/language-ext/
- neonsunset 2y ago
  If you do have useful feedback, please submit an issue or contribute a PR to dotnet/runtime. That’s how many performance improvements in LINQ come to be, that the post discusses.
Culonavirus 2y ago
There has to be a better way to do things than this annotation noodle soup. My eyes are bleeding every time I look at modern Net code.
- bob1029 2y ago
  Those attributes correspond to the benchmarking library used in the article. Unit testing & benchmarking code does typically look kind of like a plate of spaghetti. That said, you would never get a PR through me that does this in the actual business logic. You can use things like AspNetCore without touching a single attribute if you really don't want to.
- recursive 2y ago
  What .net code are you looking at? I rarely use attributes at all.
zija 2y ago
I don't understand why the dotnet team doesn't invest more resources and time into tools: doctests and documentation generation, better and faster unit tests that I can write next to the actual code, easy access to the source code, no need to decompile a Dll to see it after pressing F12, a central place for packages and documentation like in pkg.go.dev or docs.rs . Most packages in nuget have no documentation at all, or only in readme format on GitHub, or a short Wiki on this page. Other environments like Rust, Golang, Java, Python are light years ahead when it comes to these aspects.
- sibit 2y ago
  A part of me jokes that Micro$oft only invested in OpenAI because it's the only sensible way to browse .NET/Nuget package(s) documentation. The other part knows it's dreadfully true.
  - zija 2y ago
    Small F# community have https://fsprojects.github.io/FSharp.Formatting/ https://fsprojects.github.io/FSharp.Formatting/ that is a lot better than docfx, it has markdown support, notebook suport, script output embedding. C# has nothing like that, xml comments and raw documentation that has no value
- Merad 2y ago
  Sandcastle Helper File Builder has been around forever and started as an internal MS project IIRC, but for some reason few libraries use it. https://github.com/EWSoftware/SHFB https://github.com/EWSoftware/SHFB
  - progmetaldev 2y ago
    I used to use this quite a bit after it was introduced, but CHM files are a bit unwieldy for documentation, at least for myself. Having integrated search is nice, but the windowed interface, and being tied to Windows (AFAIK, maybe there are readers for other operating systems) I believe will keep this project from being used. I always found it to be decent for more advanced developers, but felt there was something missing for more junior developers that needed documentation.

203 points by olvy0 2y ago

LINQ is so fucking useful and well designed feature of .NET ecosystem that it is unreal when you gotta use lang which doesnt have such a thing. C# design team is/was unparalleled

pjmlp 2y ago
LINQ is largely based on FP stuff, also how Smalltalk collections work. It is relatively easy to find similar capabilities in most languages nowadays, unless one is stuck on Go, C and similar.
- sanex 2y ago
  Do you know an equivalent for Linw to EF in kotlin or Java because I have not found it.
  - stanac 2y ago
    Those are linq expressions. They are indeed wonderful. You get an abstract tree from which you can create SQL or API commands to access the data source. I remember in the early days (.NET 3.5?) there were multiple examples of LINQ2X like Linq2Csv, Linq2Rss Linq2Drobox (I'm paraphrasing, I don't remember actual examples, but it was wild). There is also relinq library which transforms linq expressions into expressions which are easier to understand/use.
  - pjmlp 2y ago
    Rather quick search, as I am more of a myBatis person, Java: https://www.jooq.org/ https://www.jooq.org/ Kotlin: https://www.ktorm.org https://www.ktorm.org
- kumarvvr 2y ago
  Any pointers to such libraries in python?
  - pjmlp 2y ago
    itertools would be the starting point, unfortunelly Python is rather limited due to the way it only supports one line lambdas.
- apwell23 2y ago
  No it isn't easy to find similar capabitites in java, go, python, ruby. Maybe you do simulate some of this using meta programming in ruby but its certainly not 'easy to find'.
  - pjmlp 2y ago
    It certainly is, unless you are talking about SQL like syntax, which is basically syntax sugar for classical FP. And I explicitly left Go out of my list.
    - apwell23 2y ago
      >unless you are talking about SQL like syntax yes thats what linq is? https://learn.microsoft.com/en-us/dotnet/csharp/linq/ https://learn.microsoft.com/en-us/dotnet/csharp/linq/ "Language-Integrated Query (LINQ) is the name for a set of technologies based on the integration of query capabilities directly into the C# language." With LINQ, a query is a first-class language construct, just like classes, methods, and events. doing this in java is not LINQ imo List<Integer> lowNums = filter(toList(numbers), new Predicate<Integer>() { @Override public boolean apply(Integer n) { return n < 5; } });
      - high_na_euv 2y ago
        To be fair, 99% of linq usage is method chaining syntax, not query syntax
        apwell23 2y ago
        LINQ = Language Integrated Query Its even in the name. What do you mean by "method chaining is linq" ?
        ygra 2y ago
        Considering the methods live in the System.Linq namespace, I think the extension methods may also be called LINQ.
- blackoil 2y ago
  One difference with LINQ is its ubiquity. It works with database, in memory data structures, on disk files. You can use your skills/code across all the system.
  - John23832 2y ago
    It's just built on top of anything that is Iterable. If a language has first class iterator support, they could do something similar.
    - naasking 2y ago
      Mainly first-class functions I think. If you have those, you can just use fold in the core combinators.
    - mythz 2y ago
      Takes a lot more than that, LINQ providers work by accepting a LINQ Expression Syntax tree instead of an opaque function, which allows providers to inspect and traverse the Expression's AST and translate it into the data source it's implementing. This Expression AST is constructed by the compiler, not something that can be tacked on by a library later.
      - megadal 2y ago
        Yes, but I think the point is practically every high level language can already do this pretty trivially. If it's scripted you can typically just get a string representation of the function. If it's Java, JAR inspection/dynamics have been a thing for a long time. And in other languages, they usually directly support metaprogramming (like Rust) and plugging code into the compilation logic.
        mythz 2y ago
        If it were trivial you'd see LINQ-like providers implemented in "practically every high level language". Source code of the function means you have to implement the parser/lexer to convert it into a usable AST which is bad for both runtime performance and library size. Very much doubt this is available in Java, which Java ORM lets you use native Java language expression syntax to query a database?
- jiehong 2y ago
  I've found [0] for clojure, which maps the best IMO, but it also contains links to the same LINQ examples in other languages (java, kotlin, swift, elixir, python, ...). [0]: https://github.com/mythz/clojure-linq-examples https://github.com/mythz/clojure-linq-examples
  - nightski 2y ago
    The difference is many of those are dynamically typed languages. It's still useful, but a lot of the a beauty of LINQ comes from the fact that it is within a statically typed language.
- John23832 2y ago
  It's surprising that Go didn't ship with it, but given that they just added iterators, it's coming. Rust has combinators, which is the same thing. Most new languages are recognizing that functional support (even if they don't consider themselves FP languages) is necessary.
  - rw_panic0_0 2y ago
    nah it's not coming, functions like maps and filters won't come to go by design, iterators are not only about FP stuff
  - pjmlp 2y ago
    Go culture is quite clearly against this kind of niceties.
  - devjab 2y ago
    You can do functionality similar to LINQ with chaining as long as you don’t need to call a method with a generic different from the one defined. If you do need that, you’re going to have to do it without chaining. You can still do something similar but it’ll be a lot less elegant than how C# does it. It’s part of the design philosophy of Go though. They don’t want any magic. It’s similar to why they enforce explicit error handling instead of allowing you to chose between explicit and implicit. They want you to write everything near where it happens and not rely on things you can’t see. It’s probably the primary reason that Go is either hated or loved. I think it’s philosophy is great, a lot of people don’t. I have written a lot of C# over the years, so I’m a little atypical in that regard, I think most C# developers think Go is fairly inferior and in many regards they are correct. Just not in the ones that matter (come at me!). To elaborate a little on that, Go protects developers from themselves. C# is awesome when it’s written by people who know how it works, when it’s not you’ll get LINQ that runs in memory when it really shouldn’t and so on.
- whizzter 2y ago
  Yes and no, the LINQ syntax being coherent between IEnumerable<> and IQueryable<> hides a lot of good magic. IEnumerable<> is regular in-memory lambdas/streams, same what you find in many places. IQueryable<> relies on the LINQ expressions, those CAN be JIT compiled for direct execution, but the fact that they are data-objects is what allows the translation to SQL and execution on the server rather than locally and can give massive gains since processing can be done where the data lives.
  - whizzter 2y ago
    For reference, to achieve what IQueryable does with 100% normal code in JavaScript you need something like Qustar that was posted here a month ago. Regular transform code in JS (Like IEnumerable) const ids = users.filter(user => user.age<18).map(user => user.id); IQueryable like to be transformed to the server: const ids = users.filter(user => user.age.lt(18)).map(user => user.id); In C# it'd look identical, but in JS or Java this would be achieved via proxy-object hacks (the .lt() function in the filter instead of the < operator and the .id property getter for the user map to send a flag under the hood for the mapping function). https://github.com/tilyupo/qustar https://github.com/tilyupo/qustar
munchler 2y ago
If you like LINQ, you should really give F# a try.
- neonsunset 2y ago
  F# is even better. I only wish performance wasn't hit or miss around sequences/iterator expressions. Hopefully one day it will reach parity with C# through compiler improvements and/or detaching FSharp.Core from conservatively targeting NS2.0 and modernizing it.
  - emn13 2y ago
    I never really got over the order-dependent and fairly slow compile times, but it's been like 10 years since I used it for anything even slightly complex. Is F# better in this regard now, or are there accessible patterns to help deal with that?
    - munchler 2y ago
      Order-dependency is a feature, not a bug, but it does take some getting used to. If you really want to avoid it, you can now declare all functions in a module to be mutually recursive, so you can put them in any order. Cyclic dependencies between files are still prohibited, for good reason. See https://fsharpforfunandprofit.com/posts/cyclic-dependencies/ https://fsharpforfunandprofit.com/posts/cyclic-dependencies/. The F# compiler is slower than the C# compiler, but it's still more than fast enough for building large applications.
    - throwaway31424 2y ago
      [dead]
mhh__ 2y ago
Maybe you've been unlucky but LINQ didn't really seem all that interesting to me using it. It's pretty well put together but it was very hard to work out the patterns of what it was doing underneath e.g. I could not tell you now how to implement a custom IQueryable (I know where to look but couldn't tell you the rhythms of it) for some database and I am the type of person who usually does go and find that kind of thing out before using something for a "serious" project. Maybe it's just a microcosm for C# as a whole - very productive language, good design team, quite hobbled by it's weird/philistine upbringings: Bjarne said within C++ there is a simple language trying to escape, in C# you basically have a simple language buried in nouns.
- pjc50 2y ago
  > It's pretty well put together but it was very hard to work out the patterns of what it was doing underneath e.g. I could not tell you now how to implement a custom IQueryable There's a lot hidden in there, but basically they expect you to use EF. Writing an IQueryable is a similar amount of work to writing a SQL query planner. You get passed a tree of Expression objects. https://learn.microsoft.com/en-us/archive/blogs/mattwar/linq-building-an-iqueryable-provider-part-i https://learn.microsoft.com/en-us/archive/blogs/mattwar/linq...
  - SideburnsOfDoom 2y ago
    > basically they expect you to use EF. Writing an IQueryable... I don't agree. I don't feel any expectation to use EF. It would not be relevant anyway to our code. LINQ is not EF and EF is not LINQ. EF uses LINQ but not vice versa. LINQ is useful without EF. The LINQ extension methods that we use constantly are on IEnumerable<T> so EF and IQueryable is of no importance to us, but LINQ is used everywhere.
    - pjc50 2y ago
      > IQueryable is of no importance to us I was discussing IQueryable with someone else to whom it was important. In reply to > I could not tell you now how to implement a custom IQueryable (I know where to look but couldn't tell you the rhythms of it) for some database And "for some database" the default answer is "use EF" as the intermediary between LINQ queries and the database itself, rather than delving into IQueryable. LINQ-to-objects is very important and useful but I was talking about something else.
hggigg 2y ago
LINQ is a veritable footgun in any large team I find. While it's extremely powerful and really nice, it's so so so easy to blow your toes off if you don't know what you are doing. Some of my favourite screw ups I saw: * Not understanding when something is evaluated. * Not understanding the computational complexity of multiple chained operations/aggregates. * Not understanding the expectation that Single() requires exactly one of something. * Not understanding how damn hard it is to test LINQ stuff.
- high_na_euv 2y ago
  >Not understanding the expectation that Single() requires exactly one of something. Sorry, but idk how it is footgun of LINQ. It is like complaining about 0 or 1 based indexing >Not understanding how damn hard it is to test LINQ stuff. Any examples? Because I struggle to see such
- pjc50 2y ago
  How much of that is LINQ-specific and how much is just the cost of using an ORM to build queries rather than typing them out as SQL? I've never encountered testing problems with LINQ-to-objects.
- John23832 2y ago
  I sort of agree. I recently had to code splunk a bug with 3 other engineers and we all got to a relatively complex LINQ and of the 4 of us, we all had 4 different interpretations when visually inspecting. > Not understanding how damn hard it is to test LINQ stuff. I disagree with this. Just run the LINQ query on a compatible iterable.
  - hggigg 2y ago
    Regarding the testing, it's more the sheer multiplicative number of cases you have to consider on a LINQ expression.
  - SideburnsOfDoom 2y ago
    > I recently had to code splunk a bug with 3 other engineers and we all got to a relatively complex LINQ and of the 4 of us, we all had 4 different interpretations when visually inspecting. You can write hard to read code with any framework. Yes it takes effort sometimes to make linq code clear, but you should not give up on it.
highwaylights 2y ago
I’ll get started then end up on a rant but.. This is really the thing with the entire .NET stack that’s very hard to communicate. The standard library and framework design are so well thought out relative to anything else out there. More than that, the support within VS is beyond any other dev tool that exists for any other language - it’s not even particularly close. Edit-and-continue comes to mind, which despite how many times people confuse the two is not hot reload, and is wildly more productive and useful. I remember back close to 20 years ago DHH was espousing Ruby/Rails and that the concept of types at all were a code smell, and thinking “you’re just very wrong, and clearly aren’t familiar with what else is out there”. Eventually a lot of that crowd moved to Node, then to typescript, and came around. VS Enterprise (expensive as it is) had features 15 years ago that still seem magical when I show them to JS/TS folks now. IntelliTrace is one that comes to mind - there’s nothing remotely close to it’s snapshot debugging that I’ve seen anywhere else, and I’ve really looked. The big problems with the ecosystem are that the docs are exhaustive but terribly boring, and not well explained from a learning-something-for-the-first-time perspective. They also really expect that everything you do is the Microsoft way, so if you’re trying to interface your code with something like an Avalonia UI, you’re on your own. The language is absolutely wonderful though, even when used with Rider. The productivity relative to node/typescript is better enough that it crushes my soul having to go back to wrestling tsconfig and imports after working with .NET references for even small changes. So many of the little things I used to take for granted really just work, and work well. It’s just a wonderful piece of work executed over decades, held back by poor community outreach and badly written documentation.
- high_na_euv 2y ago
  Your experiences are coherent with mine. Developer experience is far ahead any other technology out there. Std lib and its API design is world class, I wish cpp had as good stdlib. Tooling is strong, especially debugger
- bob1029 2y ago
  Agree with all of this. I think the biggest problem with popularity around these tools is that they're too good. When everything just works, you have a lot more time to focus on your problem. Once you can focus on your problem, you may quickly find out you don't actually care about it. Focusing on tools is a great way to hide from this reality.
megadal 2y ago
It's just an API for JIT, basically metaprogramming. It's cool but you can definitely do a similar thing in pretty much every high level language. With scripting languages, it's all JIT :) The C# teams progress on this has been slow. Keep in mind the CIL bytecode has had such capabilities for at least 20 years now and only in the past like decade are we seeing more features and optimizations around LINQ and System.Reflection.Emit. Dynamics were extremely slow in C# and if you look at the CIL generated you see why. It's possible for example to use something like a Haxe anonymous types[1] to optimize Dynamics so that CallSite caching is way more performant. I am pretty sure in C# the only way to accept an anonymous type is as a dynamic value, so even though the type of the structure is well-defined at compile-time, it will still rely heavily on runtime reflection/DLR with no additional caching beyond what DLR does for any other dynamic type. Anyways, this leads to niche libraries being built for handling dynamic data like JSON performantly. Which leads to annoying things like .NET libraries/apps being incompatible (without some adapter) if they use for example, different JSON libraries under the hood. (See [2]). Problems like these (the lack of actually good JIT/dynamic code support) in my opinion significantly slow down the .NET ecosystems development, that's why it always feels like .NET is just catching up with features other popular languages have. To be fair though, much of C#'s lag is owed to Microsoft's contribution to .NET being mostly technical debt. Almost everything good that came out of .NET came from open source/non MS teams (like Mono). [1] - https://haxe.org/manual/types-anonymous-structure.html https://haxe.org/manual/types-anonymous-structure.html [2] - https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/how-to https://learn.microsoft.com/en-us/dotnet/standard/serializat...
- jeswin 2y ago
  > It's cool but you can definitely do a similar thing in pretty much every high level language. No. When it was release (circa 2007), very few mainstream languages embraced "Code as Data" the way C# did. In Java, there was no way to pass an expression (as an AST) to an SQL library. Which is why LINQ is so much more ergonomic than Hibernate. In C#, you could use language features you're already familiar with (such as "order.id > 100 && order.id < 200") in your queries, whereas Hibernate made you learn the framework's specific patterns (add Criteria etc etc, I don't recall now). Java just wasn't expressive enough for this. In fact, you couldn't do this even today in a language like say Python or JS. I mean, not without running it through something like babel to get an AST, and having arbitrary rules on what's code and what's data. C# had this in the spec; based on whether it was IQueryable. > Almost everything good that came out of .NET came from open source/non MS teams (like Mono). My team adopted Mono very early - like in 2005. Your statement is not true at all. C# and the Framework was a very good spec irrespective of what Open Source / Mono did, and while Mono existing might have accelerated .Net's transition into Open Source, it would have happened anyway due to the definitive swing towards Open Source in the 2000s. Linq-to-SQL, Asp.Net MVC, EF etc didn't come out of Mono.
deleted 2y ago
[deleted]

neonsunset 2y ago

Relevant section in the annual book-sized post on all performance improvements in .NET 9: https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-9/#linq https://devblogs.microsoft.com/dotnet/performance-improvemen... (for some reason HN would not allow to resubmit it, and the post was left by wayside not reaching frontpage)

dingdingdang 2y ago

OK, so since I've never actually bothered to look at what LINQ actually is.. there may be others in the same boat: "LINQ allows you to query any enumerable collections such as List<T>, Array, or Dictionary<TKey,TValue>.". Got to admit I still find the LINQ to DB passtru rather opaque even after reading up on it: https://stackoverflow.com/questions/30716776/passing-a-query-directly-through-linq-to-the-db/30716910#30716910 https://stackoverflow.com/questions/30716776/passing-a-query... Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL (in my optics anyway). Does anything library wise come close to this within the Golang ecosystem?

caseymarquis 2y ago
While LINQ does include a library of extension methods for functional programming with .NET collections (which is great), it also includes "Expression Classes". In a nutshell, this allows a user to pass a single expression lambda to a function, and the function implementor receives the abstract syntax tree for the lambda, not the lambda itself. You can then not only receive and analyze these trees, you can also manually build and compile them. This effectively allows a limited set of runtime macros within .NET.
pjc50 2y ago
> Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL. Pretty much. There's the "language integrated" version which looks a lot like SQL: var foo = new List<int> { 1, 2, 3, }; var response = from x in foo where x > 1 select x.ToString(); But that just translates to the method-orientated one which many people prefer response = foo.Where(x => x > 1).Select(x => x.ToString()); If instead of querying a List or Dictionary you query a database, using an ORM (usually Entity Framework), that will actually get converted to SQL and run on the DB.
naasking 2y ago
> Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL (in my optics anyway) It's more general and reusable than SQL, so you can map a subset of it to SQL, which is what some object-relational mappers do.

blackoil 2y ago

Anyone knows of a comprehensive book/tutorial to learn end to end web development in dotnet. Most I have found are elementary, outdated or poor quality.

DeathArrow 2y ago
I've learned by doing, but here you go. Books: C# 12 and .NET 8 - Modern Cross-Platform Development Fundamentals - Eighth Edition: Start building websites and services with ASP.NET Core 8, Blazor, and EF Core 8 by Mark J Price Web API Development with ASP.NET Core 8: Learn techniques, patterns, and tools for building high-performance, robust, and scalable web APIs by Xiaodi Yan Turorials: series of tutorials on YouTube by IAmTimCorey and Shawn Wildermuth.
littlecranky67 2y ago
That is because all the hot-and-new in .NET web development is Blazor, and it is not really popular outside of Microsoft's Blogosphere (and IMHO never will be and go down the same way as Silverlight did). The "venerable" technologies are still present in .NET 9 and still work, get maintained etc. Doing web dev in .NET nowadays for me is mostly creating HTTP/JSON/REST APIs and using whatever FE framework you like to interface it (for me that is React or NextJS). The keyword you want to search for is "ASP.NET WebApi" or - more modern - "ASP.NET minimal API". You could still do .NET MVC server-side rendering using "Razor" (the markup language in ASP.NET MVC - search for "ASP.NET MVC Razor".
- WorldMaker 2y ago
  Blazor for better and worse can't have the same ending as Silverlight did. In Server Mode it is just Razor + SignalR for HTMX-style frontends that sort of live update/"magic RPC". In Client Mode it is just Razor as a virtual DOM-ish framework running directly in Web Assembly (wasm) in the Browser. At the end of the day Client Blazor is much more like writing front end websites with Elm than writing websites for the Silverlight plugin. (Hybrid Blazor where you use both server and client possibly in the exact same assembly/project is where most of the worst Blazor mistakes lie, but those aren't the mistakes of Silverlight, those are the mistakes of ASP Classic before .NET where you can be easily confused by new versions of the which bit of this is `runat="server"` and which bit is `runat="client"` and easily and accidentally create wild RPC ravioli/spaghetti code.)

DeathArrow 2y ago

The more C# borrows from F#, the happier I am. I am awaiting for discriminated unions to finally reach C# so I can do domain modelling like a boss. :)

CharlieDigital 2y ago
You can largely already pretty easily incorporate DUs using OneOf[0] and Dunet[1]. A practical example of using this: https://chrlschn.dev/blog/2024/07/csharp-discriminated-unions-and-dotnet-channels/ https://chrlschn.dev/blog/2024/07/csharp-discriminated-union... [0] https://github.com/mcintyre321/OneOf https://github.com/mcintyre321/OneOf [1] https://github.com/domn1995/dunet https://github.com/domn1995/dunet
stult 2y ago
I'd kill for units of measure too. It makes maintaining any kind of engineering or scientific code so much easier.
- fuzzy2 2y ago
  In a past project, we used Units.NET. It worked pretty well as far as I could tell. (I was not involved with the gritty domain details.) My colleagues even extended the library where units were missing.
- Jordanpomeroy 2y ago
  In my experience units of measure in technical apps is only interesting in the UI layer as most apps I’ve worked on have a “base” system of units that all calcs are expecting. What alternate experience have you seen?
  - greggyb 2y ago
    Units of measure in F# are useful for more than unit conversion workloads. They are compile-time type information with no runtime overhead. So if you're using a lot of interrelated arrays, you can make a unit of measure for indices into just a specific type of array. This allows type-level enforcement of never using the wrong index for the wrong array. Anywhere you might want to keep categories (not category theory) of numeric values distinct from one another without intermixing, units of measure in F# can cover you.

olvy0 2y ago

For me, the most useful part of LINQ is neither the IQueryable syntax tree based extension mechanism, nor the language integrated part (which I dislike), but simply the IEnumerable extensions. Originally known somewhat confusingly as linq to objects. Those allow me to write c# in functional style, keeping the code concise. The post I submitted refers mostly to optimizations to those extension methods. This clicked for me after having learned Haskell. It also shares some of Haskell's features/pitfalls, such as laziness. There are pitfalls, sure, and honestly I wouldn't advise a team having no one somewhat experienced with basic functional idioms (including laziness) to use it. It can lead to obtuse and slow code if used indiscriminately. I try to lead by example myself.

osigurdson 2y ago
C# has quite a few easy to use things that take a while to understand. In some ways it is a bit of an "experts only" language.
- aksss 2y ago
  It’s hard to forget what you know and see through eyes of a beginner, but I definitely find C# to be a very “rich” language, in the sense of finding new features and ways of doing things the more I use it and as the incessant releases roll out. I like to think it’s still pretty easy to start with though if one is focused on the fundamentals. Lots of head room for progressively expert use, I guess.
jasonthorsness 2y ago
This is how I use LINQ as well. With some non-standard names, it has everything you need! Eric Lippert wrote a great series on monads tying it to LINQ: https://ericlippert.com/2013/04/02/monads-part-twelve/ https://ericlippert.com/2013/04/02/monads-part-twelve/
- karmakurtisaani 2y ago
  Why on earth they named map as s Select and flatmap as SelectMany, I'll never understand. Still vert useful tho. (I first learned FP with Scala, so the names from there feel the most natural to me, tbh) Edit: it just occurred to me it has to be SQL inspired or something like that.

nightski 2y ago

Whenever I work in other languages/ecosystems LINQ is the one thing that I really miss. It's just such a nice capability to have available in the standard library. Beautifully designed given the constraints it had to work within.

EVa5I7bHFq9mnYK 2y ago

It's a shame, actually, that .NET performance improvements of up to x1000 could still be found after two decades and hundreds of millions spent on development.

deleted 2y ago
[deleted]
eknkc 2y ago
Most of the time, it is not because there were too many slow things to be improved, it is mostly because they are adding more facilities to the runtime, enabling other performance improvements. For example, the ReadOnlySpan type is a recent addition to the runtime and it will allow faster iterations when used. They simply enabled Linq to use it now.
- EVa5I7bHFq9mnYK 2y ago
  ReadOnlySpan is a breakthrough innovative data structure, consisting of a pointer and a _length, that took Microsoft Corporation two decades to invent. Well, better late than never.
  - neonsunset 2y ago
    other languages do not have special pointers that can point to GC object interiors, be transparently addressed with arithmetics and cooperatively updated by GC without pinning, while also allowing to point to stack and unmanaged memory
    - EVa5I7bHFq9mnYK 2y ago
      I see, thanks. Still, x1000 performance improvement is called a bug fix.
      - neonsunset 2y ago
        I encourage you to read through this section: https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-9/#linq https://devblogs.microsoft.com/dotnet/performance-improvemen... and go through the pull requests that it references. The Nx elements figure is a speedup you would expect if you introduce a shortcut to any algorithm that can bypass doing full iteration, and >100x speed-up is what you would expect from going from interface dispatch per element (even if devirtualized behind a guard) to a bespoke vectorized implementation. Even if you happen to have irrational dislike of .NET, this is still useful learning material and perfectly applies to other languages that expose similar functionality (provided they have sufficiently good type system and generics).

issafram 2y ago

[flagged]

deleted 2y ago
[deleted]

ibejoeb 2y ago

> Some more optimizations can happen when the chain ends up with methods like Count(), First(), Last(), ElementAt() or Sum(). For instance, OrderBy(criteria).First() can be optimized to execute as Min(criteria). As useful as that might be, we should really be writing better code. This is interesting for dynamically generated chains, I suppose. But if these operations are done on bespoke code, this is kind of perverse positive reinforcement. The library itself is recognizing a sub-optimal pattern and correcting for it. I hope there's feedback, at least, that suggests improvements to the underlying code.

hakanderyal 2y ago

Beware, once you get used to using LINQ and usually work in a domain that it shines, you won't want to go back to using anything else.

jayd16 2y ago
Do not, my friends, become addicted to LINQ. It will take hold of you, and you will resent its absence.
klysm 2y ago
It’s still less powerful than something like polars

stonethrowaway 2y ago

C# is Pragmatic [0]. [0] https://news.ycombinator.com/item?id=41761346 https://news.ycombinator.com/item?id=41761346

mannycalavera42 2y ago

I have fond memories of using LINQPad That's why I guess today I work in Clojure :)

m_fayer 2y ago
It’s been consistently one of my favorite tools for a good decade. I don’t know how much Microsoft is paying them to keep it Windows-only because that’s the only reason I keep windows around, and I’m sure that goes for a number of important developers.
- stuartd 2y ago
  There’s a Mac version coming (very exciting!) https://x.com/linqpad/status/1820379112651026587 https://x.com/linqpad/status/1820379112651026587
- phito 2y ago
  I'm using NetPad on Linux. Still far away from LinqPad but it's pretty good.
iLemming 2y ago
I can relate. I used .Net at the beginning of my career, and that one was one of the few nice things. I enjoyed building in C#, until one day I realized that MSFT is more like a mental prison (apologies for the strong negative word) - mind that it's my personal feeling at that time, many things have changed positively since then, although I moved on already. Also, I too, use Clojure today - it is very nice.

louthy 2y ago

"LINQ Performance improvements" really should read: "our own List<T> implementation performance improvements". Microsoft seem to spend their time improving what they need, rather than spending time on general improvements. LINQ (the syntax, not the method extensions) needs investment. Primarily around lambda allocation and potentially on compile-time reduction of lambdas. It's time to have value-type local lambdas, or a strategy that doesn't make lambda allocation the overhead that it is. And also there really should be wildcard support (`_`) for LINQ variables by now. Which was been completely ignored when they were brought in for lambdas. It should also be possible to use a lifted-type (like IEnumerable<T>, Option<T>, etc.) as the final item in a LINQ expression, rather than `select ...`. The select adds overhead that isn't needed for certain use-cases and limits things like tail-recursive LINQ expressions. Libraries like mine that go all-in on LINQ [1], but aren't using `IEnumerable` or `IQueryable`, or any of the LINQ extensions, continually get ignored because MS are focusing purely on improving the performance of their own projects. A good example is the improved lambda inference. It was only brought forward because ASP.NET Core needed it for its minimal API. It seems like many of the features of the language/framework are driven by their own needs rather than those of the community. The absolute worst thing is the ever expanding set of 'magic methods' (like the LINQ extensions Select, SelectMany, and Where, but also GetAwaiter, and the rest). MS are adding capabilities for themselves (the compiler mostly) rather than do what is really needed and add proper higher-kinded traits to resolve the magic. So everything is weakly typed and only vaguely discoverable by the compiler :/ LINQ is one of those key differentiators between languages, yet it's been wallowing, pretty much untouched since C# 3. I think it's a crying shame that it has basically been ignored since then and, even now, they think LINQ is only useful for iterating lists. And primarily only their own list implementations. /rant Don't get me wrong, I appreciate all performance improvements, I'm sure it will help plenty of users. But the focus always seems to be narrowly targeted, which limits the potential. [1] https://github.com/louthy/language-ext/ https://github.com/louthy/language-ext/

neonsunset 2y ago
If you do have useful feedback, please submit an issue or contribute a PR to dotnet/runtime. That’s how many performance improvements in LINQ come to be, that the post discusses.

Culonavirus 2y ago

There has to be a better way to do things than this annotation noodle soup. My eyes are bleeding every time I look at modern Net code.

bob1029 2y ago
Those attributes correspond to the benchmarking library used in the article. Unit testing & benchmarking code does typically look kind of like a plate of spaghetti. That said, you would never get a PR through me that does this in the actual business logic. You can use things like AspNetCore without touching a single attribute if you really don't want to.
recursive 2y ago
What .net code are you looking at? I rarely use attributes at all.

zija 2y ago

I don't understand why the dotnet team doesn't invest more resources and time into tools: doctests and documentation generation, better and faster unit tests that I can write next to the actual code, easy access to the source code, no need to decompile a Dll to see it after pressing F12, a central place for packages and documentation like in pkg.go.dev or docs.rs . Most packages in nuget have no documentation at all, or only in readme format on GitHub, or a short Wiki on this page. Other environments like Rust, Golang, Java, Python are light years ahead when it comes to these aspects.

sibit 2y ago
A part of me jokes that Micro$oft only invested in OpenAI because it's the only sensible way to browse .NET/Nuget package(s) documentation. The other part knows it's dreadfully true.
- zija 2y ago
  Small F# community have https://fsprojects.github.io/FSharp.Formatting/ https://fsprojects.github.io/FSharp.Formatting/ that is a lot better than docfx, it has markdown support, notebook suport, script output embedding. C# has nothing like that, xml comments and raw documentation that has no value
Merad 2y ago
Sandcastle Helper File Builder has been around forever and started as an internal MS project IIRC, but for some reason few libraries use it. https://github.com/EWSoftware/SHFB https://github.com/EWSoftware/SHFB
- progmetaldev 2y ago
  I used to use this quite a bit after it was introduced, but CHM files are a bit unwieldy for documentation, at least for myself. Having integrated search is nice, but the windowed interface, and being tied to Windows (AFAIK, maybe there are readers for other operating systems) I believe will keep this project from being used. I always found it to be decent for more advanced developers, but felt there was something missing for more junior developers that needed documentation.