14 ms·
.NET 6 vs .NET 5 speedup
- Ygg2 5y agoDo be warned that .Net 6 did cause some regressions, in reflection at least https://github.com/dotnet/runtime/issues/61486 https://github.com/dotnet/runtime/issues/61486
- vadfa 5y agoIsn't reflection by definition unstable?
- GordonS 5y agoNot so it would crash the entire app with an uncatchable exception, no.
- dtech 5y agoNo. Reflection is a program accessing or modifying its own program structure. There's not need for it to be unstable, languages like Lisp, Java and I assume C# have clearly defined semantics for it.
- SideburnsOfDoom 5y agoCorrect. Perhaps parent comment meant "unstable" in the sense that it turns compile time failures into runtime ones: e.g. without reflection, if you type "customer.GetOrders()" then it either compiles or does not, whereas reflection code that finds a method called "GetOrders" can compile just fine but you won't know if it finds a method of that name or returns null, until runtime.
- phillipcarter 5y ago> There's not need for it to be unstable In practice there is though, it just depends on what you choose to take a dependency on. For example, a few years ago the C# compiler did some lambda function optimization work. This broke someone's code because they were using reflection, and ultimately depended on how lambdas were getting optimized prior to the performance improvement in the compiler. The team by-designed that regression, since they make no guarantees that you can depend on a particular implementation detail of how the compiler optimizes things. That said, when people use reflection in .NET, they're almost always programming against something that is stable and has likely worked the same way for a decade.
- phillipcarter 5y agoAlso, I can't believe I didn't mention this already: Reflection in .NET lets you dynamically invoke anything declared internal or private as well. I think it goes without saying that your code can be broken in the future if you do this.
- Ygg2 5y agoIt's the same as in Java. Java never broke this weirdly though.
- dtech 5y agoYou're confusing safety with stability. Reflection is necessarily unsafe, but not necessarily unstable. You just need to take care about keeping within bounds of guaranteed behavior. E.g. in Java calling a method through reflection is guaranteed by the language to work. 100% stable forever. Reflection also allows you to call internal JVM methods. This might or might not work depending on the JVM, making reflection an unsafe feature. It's still stable on the JVM's is works on though.
- phillipcarter 5y ago> You're confusing safety with stability. Potayto, potahto I guess.
- lvass 5y agoHow safe, audited and non-invasive is .net core by now? There's a .net program I have a VM for and that's kind of a pain. Since .net had telemetry by default, running bare on my machines was never an option, and Mono wouldn't even work.
- tester756 5y ago>Since .net had telemetry by default, running bare on my machines was never an option Why? I don't see anything weird in data collected https://docs.microsoft.com/en-us/dotnet/core/tools/telemetry#data-points https://docs.microsoft.com/en-us/dotnet/core/tools/telemetry...
- AlexanderDhoore 5y agoMedical, military, industrial... Not everything is a webapp.
- tester756 5y agoOP wrote about his private machine. Anyway, OP was worried about installing .NET because it has telemetry by default, meanwhile you can disable telemetry before running your war-app or just ship standalone? idk.
- lvass 5y agoI never said what "my machines" do. Don't assume everyone has the same lax safety necessities as you have.
- DeathArrow 5y agoQuite interesting, but I would like to also see other benchmarks as the author said that the speed is constrained by the DB, ORM and other stack choices.
- mjburgess 5y agoIn the same vein as what Linus is trying to do now at LTT, it would be useful if there was a "programming journalism" lab which builds "test benches" to create canonical benchmarks and verify the claims made on tech blogs (etc.). Eg., highly standardised docker builds, on highly standardised hardware, running popular tasks using popular libraries for each "programming tech" (eg., website, stat modelling, event system, ...).
- okl 5y agoLike pts that https://www.phoronix.com/ https://www.phoronix.com/ uses?
- gostsamo 5y agoUsually the setups and architectural choices are so different that it would make more sense for everyone to get a good stat on which functions they are calling and how much and to make a prediction on the individual stats of each function. This does not take into account caching and multithreaded scenarios, but the map can never be the territory.
- littlecranky67 5y agoThere are these: https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/
- cies 5y agoThese are some microbenchmarks at best. Still nice to know who wins, but not representative of any real work scenario.
- Hnus 5y agoWhat would be representation of some real work scenarios? I think their test suite covers wide spectrum of operations enough so to be able to draw conclusions from them. https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview#fortunes https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Proj...
- creamytaco 5y agoIs .NET relevant even on Windows these days? With MSFT's Linux embrace ever tightening, why would anyone without sunk costs invest in .NET when every alternative seems to be better (or quickly getting there)?
- Semaphor 5y ago.NET nowadays runs on Linux. Even SQL Server does.
- creamytaco 5y agoRunning on Linux doesn't tell us much in itself. Can it compete with long-established alternatives on that platform? Does it have adoption? I think the answer to both these questions is a strong no.
- Semaphor 5y agoI’m not sure if that’s what you mean, but your comments sound like "I have absolutely no idea, but I’m going with my unfounded gut feeling"
- fabian2k 5y agoWhy shouldn't it be able to compete there? It works very well on Linux and modern .NET is a pretty good platform in many regards.
- 9wzYQbTYsAIc 5y agoIt certainly has the potential for both. There are scores of Windows developers who’s skills are now transferable to Linux, Apple, and Android development (albeit with some caveats).
- kreeben 5y agoI had to raise two kids so I couldn't find time to master both the Linux and Windows ecosystems so I chose the one they use at work. These days Microsoft has made me think more carefully about choosing platform and with their new proposals, Win11 and Edge The E-commerce Browser, I'm ready to jump onto Linux but how would I program that environment, when I have almost no experience in C? For me, the answer is .net. Thank you, Microsoft, for showing me a way out.
- sp33der89 5y agoI've seen the posts about all the speedups each new version of .NET gets and I'm just wondering, was .NET just alright in performance before all this? Is that the reason they can get all these speedups? :P I'd be interested in some JVM vs .NET 6 benchmarks too, which platform to chose when. EDIT: I know about the benchmarks and I also feel like sometimes these benchmarks are really optimized in a non-idiomatic way. I would love to know how performance idiomatic Java/.NET code is and if one is to start a new project today why one would choose the JVM over .NET or when someone would chose .NET over JVM.
- Semaphor 5y agoIt’s not the same, but there is this well-known framework benchmark [0], it always had the .net frameworks close to the top. I’m guessing a lot of the speedups come from getting rid of legacy cruft. With .net core/.NET 5/6 they got rid of a lot of things compared to .NET Framework 4.8 and could play with optimizations that simply weren’t doable before. That’s just me guessing, though ;) [0]: https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/
- foepys 5y ago.NET Core also introduced some new CLR features that are incompatible with the CLR used in .NET Framework. Span<T> for example. This has happened before with .NET Framework 2, 3, 4 etc. but instead of making a .NET Framework 5 they rather made .NET Core cross-plattform and threw backwards-compatability-at-all-cost out of the window. While all .NET Framework applications (except the ones that do naughty stuff with reflection) that were compiled for .NET Framework 4.5 behave the same way on .NET 4.8, .NET (Core) got rid of this and lets developers bundle the CLR directly, giving them more leeway for incompatible changes.
- rndgermandude 5y agoIt's that in part. Here are some of my additional observations and or guesses. They invested a lot of time adding language features with compiler and runtime support to avoid e.g. heap allocations/copying, like Span<> and friends, (readonly) ref structs, in/ref/out parameters (ref and out parameters existed before but were used a lot less in the runtime), or ValueTasks to some degree. This in turn enabled a lot of potential for optimizations in the compiler (aside from essentially writing an entirely new bytecode compiler with Roslyn and entirely new JIT with RyuJIT, throwing out the crufty old compilers), in the general runtime, and in the specific runtimes/frameworks e.g. ASP.NET. Those optimizations have to be implemented first however, and more and more get implemented with each new version. I have a project I maintain that sees an almost 50% speedup from net48 to net5, and another 10-15% speedup from net5 to net6 (based on the time it takes to run the extensive test suite). It isn't even that compute heavy. From profiling it appears that a lot of these speedups are due to internal copies of data being avoided, and a lot of additional fast-paths in the runtime (e.g. fast-paths for byte-arrays or character-arrays as opposed to taking the generic array slow paths). Another thing of note is that they added a lot of `bool Try*(..., out result)`-style APIs meant to avoid exceptions and the associated handling, and switched a lot of internal code to use these functions. E.g. in the reference source of the net48 runtime I think there are still a lot of instances of try { var number = int.Parse(value); } catch { // slow path/error path } instead of the new-idiomatic .netcore and later style of if (!int.TryParse(value, out var number)) { // slow path/error path } try-catch was/is slow-ish, and throwing exceptions is too, aside from it preventing inlining by the JIT a lot of times. And while #nullable (source annotations for what is nullable or not) and associated annotations such as MaybeNullWhen() had no direct influence on how the compiler could optimize, it probably helped people a lot writing correct code and as a side effect a lot more code became compile-time provable non-nullable which enabled further optimizations e.g. generating code that skips redundant null checks.
- Rochus 5y agoThe speed-up very much depents on the (micro)benchmark in use. I did some measurements using the Are-we-fast-yet benchmark suite which includes both micro and larger benchmarks and got an overall speed-up (geometric mean of factors) of only 2% on x86 and even a little speed-down on x64. See https://www.quora.com/Is-the-Mono-CLR-really-slower-than-CoreCLR/answer/Rochus-Keller https://www.quora.com/Is-the-Mono-CLR-really-slower-than-Cor... and http://software.rochus-keller.ch/Are-we-fast-yet_results_windows_x64.pdf http://software.rochus-keller.ch/Are-we-fast-yet_results_win... for the details.
- jmkni 5y agoWould be interested to see this benchmark on M1 (Apple Silicon), are there instructions on how to run it on that?
- Rochus 5y agoIf you want to compare different .NET versions running natively on M1 such versions must be available as a precondition. If so, just download e.g. http://software.rochus-keller.ch/Are-we-fast-yet_CLI_2021-08-28.zip http://software.rochus-keller.ch/Are-we-fast-yet_CLI_2021-08..., update the included runtimeconfig.json file to the .NET version in use and run it (dotnet Main.exe).
- jmkni 5y agoOk nice, will give that a go, cheers :)
- rndgermandude 5y agoYou're cross-compiling from the Oberon+ language to CLR IL bytecode using your own compiler... This isn't exactly something a lot of people would do. Most people would write more or less idiomatic C# and have the "official" compiler (Roslyn) produce the byte code. What I am saying, I guess, is that I am not quite sure how much of your benchmark results come down to the quality of IL your custom compiler spits out.
- kolleykibber 5y agoAre there any figures available for .net usage outside of large organisations?
- notanaverageman 5y agoNote that most of the performance improvements come from PGO, which is enabled with following environment variables. PGO is not enabled in .NET 6 by default, but will be in .NET 7 IIRC. set DOTNET_ReadyToRun=0 set DOTNET_TieredPGO=1 set DOTNET_TC_QuickJitForLoops=1 Here are my own benchmarks from a CPU intensive application without any IO and already optimized for allocations. Application runs a task graph either serially or in parallel. .NET 5 -------------------------- | Method | Mean | |------------ |---------:| | RunParallel | 473.4 us | | Run | 513.5 us | .NET 6 -------------------------- | Method | Mean | |------------ |---------:| | RunParallel | 452.5 us | | Run | 499.8 us | .NET 6 PGO -------------------------- | Method | Mean | |------------ |---------:| | RunParallel | 381.8 us | | Run | 412.2 us | .NET 5 - .NET 6 -> ~5% .NET 5 - .NET 6 PGO -> ~20% Here is what I learned from micro-optimizing a .NET application: - Use BenchmarkDotNet[0] for general measurements and Visual Studio profiler tools for detailed inspection. They help a lot. - Memory allocations matter. Using capturing lambdas, LINQ, even foreach on interfaces introduce allocations and slows down the application. You can use ClrHeapAllocationAnalyzer[1] to find these hidden allocations. - Using abstractions with interfaces and casting back to concrete types cause some overhead, though PGO will probably eliminate most of these. - Use LINQ cautiously as its variants are mostly slower than explicit coding. E.g. .Any() vs .Count == 0 - Checking Logger.IsEnabled() before calling Logger.Debug() etc. helps a lot. You can even automate this with Fody [2], but it breaks Edit&Continue and possibly .NET Hot Reload too, so it may hinder your productivity. [0] https://github.com/dotnet/BenchmarkDotNet https://github.com/dotnet/BenchmarkDotNet [1] https://github.com/microsoft/RoslynClrHeapAllocationAnalyzer https://github.com/microsoft/RoslynClrHeapAllocationAnalyzer [2] https://github.com/jorisdebock/LoggerIsEnabled.Fody https://github.com/jorisdebock/LoggerIsEnabled.Fody
- vips7L 5y agoFYI your chart is very unreadable on mobile.
- notanaverageman 5y agoThanks, I removed unnecessary parts. It should be better now.
- btrask 5y agoI might be the last person to realize this, but did Microsoft name it .NET because they already had COM?
- ygra 5y ago.NET actually was known as COM+ for a time before public release. Environment variables for tweaking internal behaviour still retain that moniker.
- randerson 5y agoCOM+ was basically Distributed COM, and was available for years before .NET. .NET Framework was implemented built on existing Win32 and COM/COM+ calls though, which is why you might see that.
- DenisM 5y agoDistributed COM was known as DCOM, COM+ became known as CLR.
- contextfree 5y agoYou're both sort of right. There was something that was released under the name COM+, which was a bunch of services on top of DCOM. But what became the .NET CLR was also internally called COM+ (or part of COM+?) under development. see https://wiki.c2.com/?ComPlus https://wiki.c2.com/?ComPlus I think what happened might have been similar to what ended up happening with the .NET name later - there was a name associated with an umbrella strategy, a bunch of different technical components were under development associated with that strategy, but only some of them were released before the strategy changed again, while others were repurposed/repositioned to be part of the new strategy. This is a common pattern with Microsoft product/feature naming and I think it's one of the reasons everyone including Microsoft developer relations people routinely comment that Microsoft "not good at naming things". It's continuing now with UWP and WinRT, where those names are actually used to refer to a bunch of different things that were once part of a now-defunct Windows strategy - some of these things are now deprecated, while others (like the WinRT core language interop model) are still the basis of most new Windows API development, but this is very confusing to developers because of their association with the abandoned overall UWP strategy
- dgellow 5y agoThe library benchmarked in the article is Stl.Fusion: https://github.com/servicetitan/Stl.Fusion https://github.com/servicetitan/Stl.Fusion. I've only learned about it today, and the documentation is a bit messy, but that seems to be a really interesting project. The author describes it as a .Net library to quickly develop efficient, distributed, real-time web applications.
- tluyben2 5y agoWe are testing on .NET 6 now with a large LoC monolithic asp.net system and the results indeed again have improved. We already rewrote a lot when we moved to .net 5 core to be more idiomatic so I guess those things were optimized more. It is not a huge jump but definitely nice work! Edit; will try to post some numbers when all tests succeed; it is closed source but for a large (millions LoC) codebase I think it is nice to see how it performs under the same conditions compared to our current prod.
- mikewarot 5y agoWhat does .NET actually do?
- cyral 5y agoIt is basically the standard library for C#/F#/VB.NET applications.
- mikewarot 5y agoI've never had to use it in my code, but I've had plenty of problems with its use in applications. It always seemed to me that it was Microsoft's attempt to lock people into Windows forever.
- hu3 5y ago.NET Linux support was announced a few years ago. > It always seemed to me that it was Microsoft's attempt to lock people into Windows forever.
- mmphosis 5y agoI have a Windows .exe program that requires .NET and I have yet to figure out how to run this in Wine. I tried installing various versions of .NET in Wine but it was a nightmare. I also tried installing Mono the .NET for Linux? Mono didn't help getting the .exe running. I am only locked into Windows by this one .exe that I need to run.
- cyral 5y ago> It always seemed to me that it was Microsoft's attempt to lock people into Windows forever. It has had first-class support for Linux and MacOS for the past five years so that certainly isn't the case these days. I actually develop C# applications on Mac and run them in production on Ubuntu, no Windows involved in the toolchain anymore.
- Rapzid 5y ago.NET is an umbrella term for the platform ecosystem that includes, among other things: * Common Language Runtime(aka CLR, aka runtime) and its JIT compiler RyuJIT * The C# language and it's compiler Roslyn * The base class library and framework class libraries(aka BCL and FCL) * ASP.NET the flagship web application framework * F# the platforms flagship functional language * etc etc It's all ".NET"
- jmnicolas 5y agoI was so happy with upgrading one of my app to .net 6: I saw perfs gains from 10 minutes execution time on .net 4.8 to 1'30 minutes on .net 6. Then my boss reminded me that we had new hypervisors with SSD (the old one had still spinning platters) so now I'm not so sure the .net 6 upgrade really made my app faster.
- mihular 5y agoHighly unlikely that you would gain that much just by upgrading. What kind of an app is anyway?
- Shadonototra 5y agoimprovements for typical benchmark code nothing for common user code why? because benchmarking JIT code is cheating, nobody runs THE SAME code path 1_000_000 times in a row to warmup the JIT you run something here, then there, then over there, then sometimes there, this is not JIT proof and you constantly get the JIT to do work that's part of the reason Go became popular, on top of the single binary and the cloud native libraries story