7 ms·

.NET has been faster than Java on most of the benchmarkgame benchmarks for a while, since .net core 3 or so. More specifically though the JVM has tended to be

.NET has been faster than Java on most of the benchmarkgame benchmarks for a while, since .net core 3 or so. More specifically though the JVM has tended to be better about optimizing naive code than .net while .net has tended to offer more tools to do your own optimizing (unsafe, simd, value types, etc). So it would be interesting to see if the performance of naive code has improved relative to Java lately

sp33der89 5y ago
Yeah, I would love it if I could just write idiomatic code for the platform and it'd be just fast enough!
kasperni 5y ago
> .NET has been faster than Java on most of the benchmarkgame benchmarks for a while, since .net core 3 or so. And which benchmarks games are those? If I go to to the Techempower benchmark and select only C# + Java. Java comes on top in every individual category of all the benchmarks. I'm not claiming that Java is faster than .NET. Just that I don't believe one platform is significantly faster than the other. [1] https://www.techempower.com/benchmarks/#section=data-r18&hw=ph&test=json&l=zik0vx-sf https://www.techempower.com/benchmarks/#section=data-r18&hw=...
- grumpyprole 5y ago
  Such programs are often specially and painstakingly constructed to avoid all the commonly used language features that are inefficient. For example, in Java, user-defined data types are heap allocated and generic code boxes everything, even primitive types (an ArrayList of ints becomes unfortuately an array of pointers). Are these programs benchmarking typical idiomatic Java, or just some subset of the language?
  - CraigJPerry 5y ago
    >> Are these programs benchmarking typical idiomatic Java https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharp.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... c# regex redux - 1.42 seconds java regex redux - 5.31 seconds Ok... but looking at the code: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/regexredux-java-3.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... import java.io.*; import java.util.*; import java.util.concurrent.CompletableFuture; import java.util.Map.Entry; import java.util.function.*; import java.util.regex.*; import static java.util.stream.Collectors.*; ... It's only using vanilla Java features. c# ? ... using System.Runtime.InteropServices; ... Interesting, why does it need that? [DllImport("pcre2-8", EntryPoint = "pcre2_compile_8", CharSet = CharSet.Ansi)] extern static IntPtr PcreCompile(string pattern, long length, uint options, out int errorcode, out long erroroffset, IntPtr ccontext); [DllImport("pcre2-8", EntryPoint = "pcre2_jit_compile_8", CharSet = CharSet.Ansi)] extern static int PcreJitCompile(IntPtr code, uint options); [DllImport("pcre2-8", EntryPoint = "pcre2_jit_match_8", CharSet = CharSet.Ansi)] extern unsafe static int PcreJitMatch(IntPtr code, byte* subject, long length, long startoffset, int options, IntPtr match_data, IntPtr mcontext); [DllImport("pcre2-8", EntryPoint = "pcre2_match_data_create_8", CharSet = CharSet.Ansi)] extern unsafe static IntPtr PcreMatchDataCreate(uint ovecsize, IntPtr mcontext); [DllImport("pcre2-8", EntryPoint = "pcre2_get_error_message_8", CharSet = CharSet.Ansi)] extern unsafe static int PcreGetErrorMessage(int errorcode, StringBuilder buffer, long bufflen); [DllImport("pcre2-8", EntryPoint = "pcre2_get_ovector_pointer_8", CharSet = CharSet.Ansi)] extern unsafe static IntPtr PcreGetOvectorPointer(IntPtr match_data); [DllImport("pcre2-8", EntryPoint = "pcre2_substitute_8", CharSet = CharSet.Ansi)] extern unsafe static int PcreSubstitute(IntPtr code, byte* subject, long length, long startoffset, int options, IntPtr match_data, IntPtr mcontext, byte* replacement, long rlength, byte* outputbuffer, out long outlength); Aha! It's because the c# impl is really just a wrapper round a native C impl of the problem. In what world is this a useful comparison? The fastest "real" c# solution is still faster than the java one though: c# (real) - 3.1 seconds https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/regexredux-csharpcore-5.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
    - zigzag312 5y ago
      I agree. That's really not usefull comparision. They should create categories for each benchmark, like: - very naive code (shortest, most readable & easy to write code) - idiomatic code - optimized code without other-language-libs wrappers and without SIMD, single threaded - optimized code without other-language-libs wrappers and without SIMD, multi-threaded - optimized code without other-language-libs wrappers and with SIMD and/or multi-threaded - optimized code with other-language-libs wrappers allowed and any other optimization technique
      - igouy 5y ago
        You are free to take the data, create whatever categories you want and publish.
        whthf_22 5y ago
        How would that undo the unfair comparison done in the benchmark game?
        igouy 5y ago
        1. Not “unfair”. 2. We wouldn’t look at the benchmarks game if we thought an alternative presentation was more useful.
    - grumpyprole 5y ago
      I agree it's not a useful comparison. That's why I don't give much weight to statements such as "Java comes on top in every individual category of all the benchmarks".
    - igouy 5y ago
      > In what world is this a useful comparison? In this world where we also compare to C++ programs. In this world where — as you acknowledge — we also compare a C# regex-redux program that does not use a third party library. https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/regexredux-csharpcore-5.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > Interesting, why does it need that? 2 out of 10 tasks regex-redux and pidigits accept third-party libraries, the other 8 out of 10 tasks do not.
  - gameswithgo 5y ago
    Yes any benchmark will be invalid for some people, such is life. If you want to claim or know something specific you will have to so your own painstaking investigation or find someone who has done that work. benchmarkgame does not attempt to compare idiomatic solutions for languages, it is closer to a “what is the best you can do” benchmark
    - grumpyprole 5y ago
      > It is closer to a “what is the best you can do” benchmark As I suspected. So of course this tells us very little about how fast idiomatic code is relative to other languages. "The best I can do" is to invoke hand optimised assembly language, but rarely is that the right choice. A much more useful test would involve benchmarking some similar real world apps that solve the same problem.
      - igouy 5y ago
        Please show the objective rules that could be used to identify "idiomatic Java" and "idiomatic C#". Please show the objective rules to direct how comparison should be done when one languages "idiomatic" is not the same as some other languages "idiomatic" — to avoid you can write Java in any language.
  - vips7L 5y ago
    Generic boxing should be fixed “soon” if they ever release Valhalla.
  - igouy 5y ago
    Please show the objective rules that could be used to identify "typical idiomatic Java" and "typical idiomatic C#". Please show the objective rules to direct how comparison should be done when one languages "typical idiomatic" is not the same as some other languages "typical idiomatic" — to avoid you can write Java in any language.
    - naasking 5y ago
      There's some low hanging fruit, like not permitting specialized collections in Java for a set of integers. Because of type erasure these are all heap allocated in Java but not in .NET. I think there's value in benchmarks showing both the fastest you can go if you need to (specializing everything to eke out max performance), and benchmarks showing how fast you will typically go if optimizing for productivity.
      - igouy 5y ago
        > … how fast you will typically go… Why would we think that would be similar for both you and `grumpyprole`.
        naasking 5y ago
        You could probably constrain it sufficiently for some set of problems. Maybe something like: Solve problem X using the standard library associative map by elaborating the following pseudo code.
        igouy 5y ago
        https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/knucleotide-java-4.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- torginus 5y ago
  While I'm not super familiar with the Java world, none of the frameworks that have a significant advantage sound familiar to me - I'm not sure how mature are they, whereas Asp.NET is the solution for writing servers under .NET.
- zigzag312 5y ago
  > And which benchmarks games are those? https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharp.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... Be aware that many implementations on benchmarksgame are much lower-level and using all kind of performance tricks than what you would normally write.
  - merb 5y ago
    well the same for techemporer benchmarks. sorry but some of that stuff is as shady as the benchmarksgame. I really don't get it, why people don't create 100% benchmarks instead of specialized ones.
    - igouy 5y ago
      > … shady… The benchmarks game puts the source code under bright light, a couple of clicks from the measurements. That's why people have noticed differences and commented.
  - igouy 5y ago
    That may be true of you. Other readers may have a broader range of skills and face different requirements.
    - whthf_22 5y ago
      >> Other readers may have a broader range of skills Relax. When comparing anything, the comparison has to be fair. This is obviously not the case here. It's like Java is a Ford Mustang from the dealer and C# is a Camaro with a F1 engine installed.
      - igouy 5y ago
        It’s like you’re still ignoring the “Camaro from the dealer”.
- gameswithgo 5y ago
  This: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharp.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... techempower is isolated to web framework testing as far as I know
- DonHopkins 5y ago
  Java's performance hasn't really mattered since Oracle took it over. There are things MUCH worse than poor performance, and being owned by Oracle is one of them.
  - ptx 5y ago
    Ownership by Microsoft isn't great either, unless you enjoy jumping through hoops to disable their telemetry[1]. [1] https://github.com/dotnet/sdk/issues/6145 https://github.com/dotnet/sdk/issues/6145
  - pjmlp 5y ago
    The alternative being no one would have bought Sun and Java would have died as Java 6. I understand Sun ex-employees have an axe to grid with Oracle, yet no one else cared for Sun assets.
  - kaba0 5y ago
    Will this bullshit ever die? OpenJDK has the same goddamn license as the linux kernel. It is (yes, the open source codebase) developed by Oracle 98+% alone, and other vendors are just forks of this code base (including oracle jdk, which contains only trivial changes AND a paid support option, for those that need it) You can hate Oracle as much as you want but their Java division is a surprisingly adapt and capable team, doing very great job at stewarding the language.
- jayd16 5y ago
  IMO that link makes .NET look very good. Aspcore, the straight off the shelf, obvious choice, is the best performing .net server? It beats Jetty and Spring but loses to a long tail of less popular frameworks
- symbol-mason 5y ago
  You're looking at pretty old results, round 18 was in 2019. I also don't think that boutique web frameworks say much about the strength of the underlying language or runtime (e.g. look at just.js). What in the Java world is in the same maturity tier as ASP.NET is open to opinion, but at least local Java devs seem to consider Spring or Micronaut as sane defaults, and of course modern ASP.NET runs circles around those. https://www.techempower.com/benchmarks/#section=test&runid=939169a8-6a56-44ae-9e50-0e714753c445&hw=ph&test=composite&a=2 https://www.techempower.com/benchmarks/#section=test&runid=9...
- akra 5y ago
  That seems a bit misleading of a comparison IMO and only one case (JSON serialisation) when I look at their data. You are also linking to a round from two years ago which is out of date. It's also showing a lot of frameworks that are not that mature and not well used in the Java camp vs ASP.NET that is widely used, full featured, has a lot of bells and whistles and a lot of plugins available for most technologies and standards. All of which could have negatively influenced performance, even the hooks to allow them to be injected in can do so even if not enabled. The fact that a full featured web framework makes it close to the top (sometimes the top) over several rounds over many of their categories of use cases I can't discount as pretty good. i.e. Its hard to read benchmarks without context of each framework shown, the compromises they have taken, how usable it actually is for building software vs just a benchmark, what shortcuts are done in the benchmark, how idiomatic is the code, etc. https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=plaintext&l=zik0vx-sf https://www.techempower.com/benchmarks/#section=data-r20&hw=... My personal experience having worked on both platforms for several years is that Java is easier to get to an acceptable performance, but the .NET runtime when you have to put the effort in has a higher upper bound of performance. It just has more tools in the CLR to work with than the JVM (e.g. value types, proper generics, spans, and more) so you can express something with a little more mechanical sympathy. Java is left with some decisions from legacy IMO that by default hurt its performance (i.e. lots of default boxing has hurt me before especially with generics). With .NET Core and future versions I think .NET is also taking up Java's default perf area as well. YMMV but if I'm worried about performance being a risk in my project .NET gives me more tools to optimize it IMO should that risk eventuate.
Mikeb85 5y ago
> benchmarkgame benchmarks Lol look at the code. N-body for example, the C# is horrific C-in-C# code with a million optimizations (just read the comments lol), the Java code is idiomatic and not optimized at all.
- symbol-mason 5y ago
  Well the fastest C# entry for n-body looks like a translation of the C/C++ versions. It is a meaningful result that the primitives of the language allow for it to hang in that company. The absence of a Java version using numerics is unfortunate, that'd be a nice addition. A lot of the coding style seems optimized around copy-pasting the C code, e.g. trying to alias the Vector methods (Vector256.Create) to their instruction name (_mm256_set1_pd). That makes the code non-idiomatic, but it also doesn't really help performance, just makes the porting easier. The F# example is on the same runtime and a better view of using the numerics directly. As a trade-off of performance, memory, and code complexity it is actually a pretty solid balance, which I wouldn't have expected. https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/nbody-fsharpcore-6.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
  - Mikeb85 5y ago
    How is it meaningful? I can inline C code in Ruby, does that mean Ruby is as fast as C now? I'd much rather see a comparison of idiomatic code in different languages. When I choose a language to build something in I'm not thinking "How can I write C in this language?"...
    - igouy 5y ago
      Please show the objective rules that could be used to identify "idiomatic code in different languages".
    - symbol-mason 5y ago
      Inline C/ASM is literally a different language. System.Runtime.Intrinsics is a library. Just because this one benchmark used the library in a C-like style doesn't mean that's required. There are many simpler implementations in all languages. I actually like that there are multiple implementations, as this lets us estimate the benefit and complexity of adopting various optimizations. Limiting the benchmark to naive implementations would penalize languages with more broad capabilities for optimization. I'd personally prefer a benchmark limited to memory-safe implementations, though.
native_samples 5y ago
Well .NET are going in the same direction as Java - as the top comment here observes the speedups come mostly from profile guided optimization which has been Java's forte since Java 1.1 or so. And .NET PGO is off by default, still, so it's got a long way to go yet.
kaba0 5y ago
Just to add, Java now has simd in the form of the incubating Vector API. I found that it is one of the best high level low level SIMD APIs, with automatic fallback to for loops on ineligible hardware, as well as having an option for preferred lane width.

sp33der89 5y ago

Yeah, I would love it if I could just write idiomatic code for the platform and it'd be just fast enough!

kasperni 5y ago

> .NET has been faster than Java on most of the benchmarkgame benchmarks for a while, since .net core 3 or so. And which benchmarks games are those? If I go to to the Techempower benchmark and select only C# + Java. Java comes on top in every individual category of all the benchmarks. I'm not claiming that Java is faster than .NET. Just that I don't believe one platform is significantly faster than the other. [1] https://www.techempower.com/benchmarks/#section=data-r18&hw=ph&test=json&l=zik0vx-sf https://www.techempower.com/benchmarks/#section=data-r18&hw=...

grumpyprole 5y ago
Such programs are often specially and painstakingly constructed to avoid all the commonly used language features that are inefficient. For example, in Java, user-defined data types are heap allocated and generic code boxes everything, even primitive types (an ArrayList of ints becomes unfortuately an array of pointers). Are these programs benchmarking typical idiomatic Java, or just some subset of the language?
- CraigJPerry 5y ago
  >> Are these programs benchmarking typical idiomatic Java https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharp.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... c# regex redux - 1.42 seconds java regex redux - 5.31 seconds Ok... but looking at the code: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/regexredux-java-3.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... import java.io.*; import java.util.*; import java.util.concurrent.CompletableFuture; import java.util.Map.Entry; import java.util.function.*; import java.util.regex.*; import static java.util.stream.Collectors.*; ... It's only using vanilla Java features. c# ? ... using System.Runtime.InteropServices; ... Interesting, why does it need that? [DllImport("pcre2-8", EntryPoint = "pcre2_compile_8", CharSet = CharSet.Ansi)] extern static IntPtr PcreCompile(string pattern, long length, uint options, out int errorcode, out long erroroffset, IntPtr ccontext); [DllImport("pcre2-8", EntryPoint = "pcre2_jit_compile_8", CharSet = CharSet.Ansi)] extern static int PcreJitCompile(IntPtr code, uint options); [DllImport("pcre2-8", EntryPoint = "pcre2_jit_match_8", CharSet = CharSet.Ansi)] extern unsafe static int PcreJitMatch(IntPtr code, byte* subject, long length, long startoffset, int options, IntPtr match_data, IntPtr mcontext); [DllImport("pcre2-8", EntryPoint = "pcre2_match_data_create_8", CharSet = CharSet.Ansi)] extern unsafe static IntPtr PcreMatchDataCreate(uint ovecsize, IntPtr mcontext); [DllImport("pcre2-8", EntryPoint = "pcre2_get_error_message_8", CharSet = CharSet.Ansi)] extern unsafe static int PcreGetErrorMessage(int errorcode, StringBuilder buffer, long bufflen); [DllImport("pcre2-8", EntryPoint = "pcre2_get_ovector_pointer_8", CharSet = CharSet.Ansi)] extern unsafe static IntPtr PcreGetOvectorPointer(IntPtr match_data); [DllImport("pcre2-8", EntryPoint = "pcre2_substitute_8", CharSet = CharSet.Ansi)] extern unsafe static int PcreSubstitute(IntPtr code, byte* subject, long length, long startoffset, int options, IntPtr match_data, IntPtr mcontext, byte* replacement, long rlength, byte* outputbuffer, out long outlength); Aha! It's because the c# impl is really just a wrapper round a native C impl of the problem. In what world is this a useful comparison? The fastest "real" c# solution is still faster than the java one though: c# (real) - 3.1 seconds https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/regexredux-csharpcore-5.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
  - zigzag312 5y ago
    I agree. That's really not usefull comparision. They should create categories for each benchmark, like: - very naive code (shortest, most readable & easy to write code) - idiomatic code - optimized code without other-language-libs wrappers and without SIMD, single threaded - optimized code without other-language-libs wrappers and without SIMD, multi-threaded - optimized code without other-language-libs wrappers and with SIMD and/or multi-threaded - optimized code with other-language-libs wrappers allowed and any other optimization technique
    - igouy 5y ago
      You are free to take the data, create whatever categories you want and publish.
      - whthf_22 5y ago
        How would that undo the unfair comparison done in the benchmark game?
        igouy 5y ago
        1. Not “unfair”. 2. We wouldn’t look at the benchmarks game if we thought an alternative presentation was more useful.
  - grumpyprole 5y ago
    I agree it's not a useful comparison. That's why I don't give much weight to statements such as "Java comes on top in every individual category of all the benchmarks".
  - igouy 5y ago
    > In what world is this a useful comparison? In this world where we also compare to C++ programs. In this world where — as you acknowledge — we also compare a C# regex-redux program that does not use a third party library. https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/regexredux-csharpcore-5.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > Interesting, why does it need that? 2 out of 10 tasks regex-redux and pidigits accept third-party libraries, the other 8 out of 10 tasks do not.
- gameswithgo 5y ago
  Yes any benchmark will be invalid for some people, such is life. If you want to claim or know something specific you will have to so your own painstaking investigation or find someone who has done that work. benchmarkgame does not attempt to compare idiomatic solutions for languages, it is closer to a “what is the best you can do” benchmark
  - grumpyprole 5y ago
    > It is closer to a “what is the best you can do” benchmark As I suspected. So of course this tells us very little about how fast idiomatic code is relative to other languages. "The best I can do" is to invoke hand optimised assembly language, but rarely is that the right choice. A much more useful test would involve benchmarking some similar real world apps that solve the same problem.
    - igouy 5y ago
      Please show the objective rules that could be used to identify "idiomatic Java" and "idiomatic C#". Please show the objective rules to direct how comparison should be done when one languages "idiomatic" is not the same as some other languages "idiomatic" — to avoid you can write Java in any language.
- vips7L 5y ago
  Generic boxing should be fixed “soon” if they ever release Valhalla.
- igouy 5y ago
  Please show the objective rules that could be used to identify "typical idiomatic Java" and "typical idiomatic C#". Please show the objective rules to direct how comparison should be done when one languages "typical idiomatic" is not the same as some other languages "typical idiomatic" — to avoid you can write Java in any language.
  - naasking 5y ago
    There's some low hanging fruit, like not permitting specialized collections in Java for a set of integers. Because of type erasure these are all heap allocated in Java but not in .NET. I think there's value in benchmarks showing both the fastest you can go if you need to (specializing everything to eke out max performance), and benchmarks showing how fast you will typically go if optimizing for productivity.
    - igouy 5y ago
      > … how fast you will typically go… Why would we think that would be similar for both you and `grumpyprole`.
      - naasking 5y ago
        You could probably constrain it sufficiently for some set of problems. Maybe something like: Solve problem X using the standard library associative map by elaborating the following pseudo code.
        igouy 5y ago
        https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/knucleotide-java-4.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
torginus 5y ago
While I'm not super familiar with the Java world, none of the frameworks that have a significant advantage sound familiar to me - I'm not sure how mature are they, whereas Asp.NET is the solution for writing servers under .NET.
zigzag312 5y ago
> And which benchmarks games are those? https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharp.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... Be aware that many implementations on benchmarksgame are much lower-level and using all kind of performance tricks than what you would normally write.
- merb 5y ago
  well the same for techemporer benchmarks. sorry but some of that stuff is as shady as the benchmarksgame. I really don't get it, why people don't create 100% benchmarks instead of specialized ones.
  - igouy 5y ago
    > … shady… The benchmarks game puts the source code under bright light, a couple of clicks from the measurements. That's why people have noticed differences and commented.
- igouy 5y ago
  That may be true of you. Other readers may have a broader range of skills and face different requirements.
  - whthf_22 5y ago
    >> Other readers may have a broader range of skills Relax. When comparing anything, the comparison has to be fair. This is obviously not the case here. It's like Java is a Ford Mustang from the dealer and C# is a Camaro with a F1 engine installed.
    - igouy 5y ago
      It’s like you’re still ignoring the “Camaro from the dealer”.
gameswithgo 5y ago
This: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharp.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... techempower is isolated to web framework testing as far as I know
DonHopkins 5y ago
Java's performance hasn't really mattered since Oracle took it over. There are things MUCH worse than poor performance, and being owned by Oracle is one of them.
- ptx 5y ago
  Ownership by Microsoft isn't great either, unless you enjoy jumping through hoops to disable their telemetry[1]. [1] https://github.com/dotnet/sdk/issues/6145 https://github.com/dotnet/sdk/issues/6145
- pjmlp 5y ago
  The alternative being no one would have bought Sun and Java would have died as Java 6. I understand Sun ex-employees have an axe to grid with Oracle, yet no one else cared for Sun assets.
- kaba0 5y ago
  Will this bullshit ever die? OpenJDK has the same goddamn license as the linux kernel. It is (yes, the open source codebase) developed by Oracle 98+% alone, and other vendors are just forks of this code base (including oracle jdk, which contains only trivial changes AND a paid support option, for those that need it) You can hate Oracle as much as you want but their Java division is a surprisingly adapt and capable team, doing very great job at stewarding the language.
jayd16 5y ago
IMO that link makes .NET look very good. Aspcore, the straight off the shelf, obvious choice, is the best performing .net server? It beats Jetty and Spring but loses to a long tail of less popular frameworks
symbol-mason 5y ago
You're looking at pretty old results, round 18 was in 2019. I also don't think that boutique web frameworks say much about the strength of the underlying language or runtime (e.g. look at just.js). What in the Java world is in the same maturity tier as ASP.NET is open to opinion, but at least local Java devs seem to consider Spring or Micronaut as sane defaults, and of course modern ASP.NET runs circles around those. https://www.techempower.com/benchmarks/#section=test&runid=939169a8-6a56-44ae-9e50-0e714753c445&hw=ph&test=composite&a=2 https://www.techempower.com/benchmarks/#section=test&runid=9...
akra 5y ago
That seems a bit misleading of a comparison IMO and only one case (JSON serialisation) when I look at their data. You are also linking to a round from two years ago which is out of date. It's also showing a lot of frameworks that are not that mature and not well used in the Java camp vs ASP.NET that is widely used, full featured, has a lot of bells and whistles and a lot of plugins available for most technologies and standards. All of which could have negatively influenced performance, even the hooks to allow them to be injected in can do so even if not enabled. The fact that a full featured web framework makes it close to the top (sometimes the top) over several rounds over many of their categories of use cases I can't discount as pretty good. i.e. Its hard to read benchmarks without context of each framework shown, the compromises they have taken, how usable it actually is for building software vs just a benchmark, what shortcuts are done in the benchmark, how idiomatic is the code, etc. https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=plaintext&l=zik0vx-sf https://www.techempower.com/benchmarks/#section=data-r20&hw=... My personal experience having worked on both platforms for several years is that Java is easier to get to an acceptable performance, but the .NET runtime when you have to put the effort in has a higher upper bound of performance. It just has more tools in the CLR to work with than the JVM (e.g. value types, proper generics, spans, and more) so you can express something with a little more mechanical sympathy. Java is left with some decisions from legacy IMO that by default hurt its performance (i.e. lots of default boxing has hurt me before especially with generics). With .NET Core and future versions I think .NET is also taking up Java's default perf area as well. YMMV but if I'm worried about performance being a risk in my project .NET gives me more tools to optimize it IMO should that risk eventuate.

Mikeb85 5y ago

> benchmarkgame benchmarks Lol look at the code. N-body for example, the C# is horrific C-in-C# code with a million optimizations (just read the comments lol), the Java code is idiomatic and not optimized at all.

symbol-mason 5y ago
Well the fastest C# entry for n-body looks like a translation of the C/C++ versions. It is a meaningful result that the primitives of the language allow for it to hang in that company. The absence of a Java version using numerics is unfortunate, that'd be a nice addition. A lot of the coding style seems optimized around copy-pasting the C code, e.g. trying to alias the Vector methods (Vector256.Create) to their instruction name (_mm256_set1_pd). That makes the code non-idiomatic, but it also doesn't really help performance, just makes the porting easier. The F# example is on the same runtime and a better view of using the numerics directly. As a trade-off of performance, memory, and code complexity it is actually a pretty solid balance, which I wouldn't have expected. https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/nbody-fsharpcore-6.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- Mikeb85 5y ago
  How is it meaningful? I can inline C code in Ruby, does that mean Ruby is as fast as C now? I'd much rather see a comparison of idiomatic code in different languages. When I choose a language to build something in I'm not thinking "How can I write C in this language?"...
  - igouy 5y ago
    Please show the objective rules that could be used to identify "idiomatic code in different languages".
  - symbol-mason 5y ago
    Inline C/ASM is literally a different language. System.Runtime.Intrinsics is a library. Just because this one benchmark used the library in a C-like style doesn't mean that's required. There are many simpler implementations in all languages. I actually like that there are multiple implementations, as this lets us estimate the benefit and complexity of adopting various optimizations. Limiting the benchmark to naive implementations would penalize languages with more broad capabilities for optimization. I'd personally prefer a benchmark limited to memory-safe implementations, though.

native_samples 5y ago

Well .NET are going in the same direction as Java - as the top comment here observes the speedups come mostly from profile guided optimization which has been Java's forte since Java 1.1 or so. And .NET PGO is off by default, still, so it's got a long way to go yet.

kaba0 5y ago

Just to add, Java now has simd in the form of the incubating Vector API. I found that it is one of the best high level low level SIMD APIs, with automatic fallback to for loops on ineligible hardware, as well as having an option for preferred lane width.