3 ms·

Well the amount of tricks that basically nobody can implement in production that those implementations do to get the numbers that they do is ridiculous. Meaning

by EraYaN 2y ago

Well the amount of tricks that basically nobody can implement in production that those implementations do to get the numbers that they do is ridiculous. Meaning the numbers they get are wildly optimistic at best and misleading at worst. If you take the standard template for any of these frameworks (both Java and C# and any other language) and you add authentication etc, the real performance will be 5-10% of the numbers reported in those benchmarks. Have a look through some of the weirdness in the implementations it's wild (and sometimes educational). The .NET team especially has done stuff specifically to get faster on those benchmarks.

neonsunset 2y ago
I second this sentiment. If anything, I genuinely think that the way .NET's TechEmpower submissions look does more damage than good. BenchmarksGame offers a much better close-up comparison by having much simpler submissions that mostly consist of code that you could totally see yourself write in a regular setting. .NET is perfectly capable of standing on its own, and if there are specific areas that need improvement - this should serve as a push to further improve DB driver implementations and make ASP.NET Core more robust against various feature configurations. It is already much, much faster than Spring which is a good start, but it could be pushed further. I'd like to note that neither Go nor Java are viable for high-performance programming in a way that C# is. Neither gives you the required low-level access, performance oriented APIs, ability to use zero-cost abstractions and platform control you get to have in .NET. You can get far with both, but not C++/Rust-far the way you can with C#.
- Ygg2 2y ago
  > BenchmarksGame offers a much better close-up comparison by having much simpler submissions that mostly consist of code that you could totally see yourself write in a regular setting Yeah, except if you are working on Web servers the quality of the framework and its supporting libraries is much more important than what code could theoretically achieve. What is the point of being able to 200 mph when you only ever drive up to 30mph. > Neither gives you the required low-level access, performance oriented APIs, ability to use zero-cost abstractions. Java is working on high performance abstractions, see Vector API (Simd) and project Valhalla (custom primitive types). Sure C# has a theoretical leg up (for which it paid dearly by causing backwards incompatibility with reified generics) but most of the libraries don't use low-level access or SIMD optimizations or what not.
dahauns 2y ago
> Have a look through some of the weirdness in the implementations it's wild (and sometimes educational). The .NET team especially has done stuff specifically to get faster on those benchmarks. Could you give me a pointer or two? I wondered about that myself, especially considering the massive improvement from "old" .NET to the core/kestrel based solutions - but a quick browsing a while ago mostly left me astonished how...well, for lack of a better word, banal most of the code was. Agreed though, lack of all kinds of layers like auth, orm etc. are sadly a drawback of these kinds of benchmarks, if understandable - it would make comparability even trickier and has the danger of the comparison matrix of systems/frameworks/libraries exploding in size. But yeah, would be nice datapoints to have. :)
- EraYaN 2y ago
  They don't even use Razor Pages but a custom RazorSlices package to do the templating [1]. Yes, that is much faster because it removes MVC and a ton of infrastructure but it's also kind of gross. Also the use of stuff like UnsafePreferInlineScheduling has some downsides (running application code on the IO thread) and honestly I'd never use in production. The custom BufferWriter stuff is pretty neat though, although also not really something most people will reach for. And there is more, like the caching of StringBuilders etc. But it also doesn't use the actual HTTP server to build headers, but they just dump a string into the socket [2], feels a bit unrealistic to me. In general the BenchmarkApplication class [3] is full of non-standard stuff that you'd normally let the framework handle. [1] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/CSharp/aspnetcore/src/Platform/Templates/FortunesUtf8.cshtml https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast... [2] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/CSharp/aspnetcore/src/Platform/BenchmarkApplication.Fortunes.cs#L25 https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast... [3] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/CSharp/aspnetcore/src/Platform/BenchmarkApplication.cs https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
  - dahauns 2y ago
    Thanks a lot for the input, that's quite enlightening - seems like I have been browsing everything but the "Platform" target... Puh, yeah, I see what you mean, much the stuff in [2] and [3] is rather...bespoke, especially compared to the minimal and mvc targets. Not really what I'd consider "realistic" as per the benchmark's definition. But TBH, I wouldn't consider [1] gross, on the contrary - simple, fast, lightweight Razor templating without other MVC (or other external) dependencies isn't that unusual a use case and something I've often thought ASP.NET Core was missing (even Razor Pages feel like overkill if you just want to quickly generate some dynamic HTML).

by EraYaN 2y ago

neonsunset 2y ago

I second this sentiment. If anything, I genuinely think that the way .NET's TechEmpower submissions look does more damage than good. BenchmarksGame offers a much better close-up comparison by having much simpler submissions that mostly consist of code that you could totally see yourself write in a regular setting. .NET is perfectly capable of standing on its own, and if there are specific areas that need improvement - this should serve as a push to further improve DB driver implementations and make ASP.NET Core more robust against various feature configurations. It is already much, much faster than Spring which is a good start, but it could be pushed further. I'd like to note that neither Go nor Java are viable for high-performance programming in a way that C# is. Neither gives you the required low-level access, performance oriented APIs, ability to use zero-cost abstractions and platform control you get to have in .NET. You can get far with both, but not C++/Rust-far the way you can with C#.

Ygg2 2y ago
> BenchmarksGame offers a much better close-up comparison by having much simpler submissions that mostly consist of code that you could totally see yourself write in a regular setting Yeah, except if you are working on Web servers the quality of the framework and its supporting libraries is much more important than what code could theoretically achieve. What is the point of being able to 200 mph when you only ever drive up to 30mph. > Neither gives you the required low-level access, performance oriented APIs, ability to use zero-cost abstractions. Java is working on high performance abstractions, see Vector API (Simd) and project Valhalla (custom primitive types). Sure C# has a theoretical leg up (for which it paid dearly by causing backwards incompatibility with reified generics) but most of the libraries don't use low-level access or SIMD optimizations or what not.

dahauns 2y ago

> Have a look through some of the weirdness in the implementations it's wild (and sometimes educational). The .NET team especially has done stuff specifically to get faster on those benchmarks. Could you give me a pointer or two? I wondered about that myself, especially considering the massive improvement from "old" .NET to the core/kestrel based solutions - but a quick browsing a while ago mostly left me astonished how...well, for lack of a better word, banal most of the code was. Agreed though, lack of all kinds of layers like auth, orm etc. are sadly a drawback of these kinds of benchmarks, if understandable - it would make comparability even trickier and has the danger of the comparison matrix of systems/frameworks/libraries exploding in size. But yeah, would be nice datapoints to have. :)

EraYaN 2y ago
They don't even use Razor Pages but a custom RazorSlices package to do the templating [1]. Yes, that is much faster because it removes MVC and a ton of infrastructure but it's also kind of gross. Also the use of stuff like UnsafePreferInlineScheduling has some downsides (running application code on the IO thread) and honestly I'd never use in production. The custom BufferWriter stuff is pretty neat though, although also not really something most people will reach for. And there is more, like the caching of StringBuilders etc. But it also doesn't use the actual HTTP server to build headers, but they just dump a string into the socket [2], feels a bit unrealistic to me. In general the BenchmarkApplication class [3] is full of non-standard stuff that you'd normally let the framework handle. [1] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/CSharp/aspnetcore/src/Platform/Templates/FortunesUtf8.cshtml https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast... [2] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/CSharp/aspnetcore/src/Platform/BenchmarkApplication.Fortunes.cs#L25 https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast... [3] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/CSharp/aspnetcore/src/Platform/BenchmarkApplication.cs https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
- dahauns 2y ago
  Thanks a lot for the input, that's quite enlightening - seems like I have been browsing everything but the "Platform" target... Puh, yeah, I see what you mean, much the stuff in [2] and [3] is rather...bespoke, especially compared to the minimal and mvc targets. Not really what I'd consider "realistic" as per the benchmark's definition. But TBH, I wouldn't consider [1] gross, on the contrary - simple, fast, lightweight Razor templating without other MVC (or other external) dependencies isn't that unusual a use case and something I've often thought ASP.NET Core was missing (even Razor Pages feel like overkill if you just want to quickly generate some dynamic HTML).