4 ms·

Unfortunately, there are a few issues with this quora post. First it is outdated - .NET is on an annual cadence and statements true for 3.1 or 5 have no impact

Unfortunately, there are a few issues with this quora post. First it is outdated - .NET is on an annual cadence and statements true for 3.1 or 5 have no impact on user experience today, second it focuses on edge case differences with log scale rather than the full picture. Also note that compiler improves significantly with each version - look at the difference between 3.1, 5 and 6. The differences between 6 and 8 are way more significant (DynamicPGO light-up, physical promotion for structs, etc.). And third there is more to the story than microbenchmarks. Again, there is a huge rift in performance between Mono and CoreCLR, and there are classes of optimizations Mono does not and will not do, like devirtualization. All optimization work goes into CoreCLR for multiple ISAs. Mono does not support AVX2 and AVX512, which core routines in .NET CoreLib rely on if available. It doesn't have a fancy if-conversion pass to merge branches and comparisons that CoreCLR does for ARM64, write barrier optimizations, etc etc. This is not a myth but what anyone who works closely with .NET or is a part of .NET teams will tell you. For example, look at https://aka.ms/aspnet/benchmarks https://aka.ms/aspnet/benchmarks select mono tab and pick coreclr and fortunes - you'll notice that Mono is more than five times slower than CoreCLR. And it is the Mono flavour which also sees performance improvements for the features that .NET adds or optimizations focused on targets served by Mono. Also, the independent build is not independent - it limps along Unity Technologies contributions until Unity finishes its move to CoreCLR, which they are in the process of.

Rochus 2y ago
The post is good enough, using recognized methods and allowing reproducability. It considers different CoreCLR and Mono versions, and there is no reason to assume big performance differences since. The raw data is referenced, you can use a different scale if you prefer, but the information is the same. If you are interested, take a look at the Are-we-fast-yet benchmark suite to see that it is well defined, not just a bunch of random micro benchmarks, but also includes a few larger programs (e.g. to exercise the GC). And again: this is about the CLR, not the framework. I'm not interested in the .NET framework implementation. The build works well on all platforms without any dependencies and without downloading and compiling several gigs of stuff.
- neonsunset 2y ago
  I am talking about CoreCLR, which has nothing to do with .NET Framework. It has tangentially something to do with .NET's standard library but its interaction with it is the same as the one Mono has. It just executes CIL assemblies, maybe AOT compiles them if you ask build system for it. The statement "there is no reason to assume big performance differences since" is naive and ignores the performance work that went into releases 7 and 8, particularly DynamicPGO which enables the whole class of new optimizations. It also ignores the steady, significant improvements in versions 3.1, 5 and 6. These are annual jumps in performance on code that had otherwise no changes. I caution other readers to take extra grain of salt and perform their own benchmarking if they do not believe my statements. Luckily, the numbers will be very easy to interpret with difference occasionally being as much as an order of magnitude in favour of CoreCLR. Mono is easier to embed which justifies select scenarios of its usage, but if there is no such specific requirement, then you should use CoreCLR. Doing otherwise means harming your project and its users in the desire to be stubborn.
  - Rochus 2y ago
    If you want to convice me, try running the referenced code on the most recent Mono and CoreCLR version and relevant architectures, and collect/report the data in an accessible way. I'm only interested in reproducible measurements, not in opinions. > Luckily, the numbers will be very easy to interpret with difference occasionally being as much as an order of magnitude in favour of CoreCLR. ... Thanks for the sales pitch.
    - neonsunset 2y ago
      It was interesting to look at how .NET deals with arbitrary assemblies from something that isn't C# or F#. In the end, I did not bother with manually linking .dll files for ILC so there will be only JIT results. In general, I do not think the CIL back-end for Oberon produces good code. It is compiler-unfriendly, ignores functions provided by .NET's CoreLib (available both under Mono and CoreCLR) to use its own poorly performing variants of them, makes everything virtual and etc. It does silly things like makes all references to a string literal into an array allocation ("test".ToCharArray()) to just do a comparison, then also treats such arrays as zero-terminated pointers despite, well, .NET arrays having length in them included already, and more. Overall, a disregard what CIL, CLI and CTS are and what they offer. What are you using .NET for if not for writing truly portable C-like? There are already pointers and manual memory management, the lowering strategy could have been much closer, and a lot of suffering is very much self-imposed. And the benchmark initially ran for so little time that it would give advantage to pure interpreter based runtime as long as it did very little, or natively compiled binaries. Surely your average program runs more than 200ms? Anyways, you can find these and a couple other notes with data and charts here: https://gist.github.com/neon-sunset/900bd310dc53dd88cbeddb5193d9104d#file-oberon-md https://gist.github.com/neon-sunset/900bd310dc53dd88cbeddb51... Geometric Mean (Mono): 1612026.37 microseconds Geometric Mean (.NET): 847482.65 microseconds Relative .NET advantage: 190.21% I did not have time to look closer in benchmarks with 1:1 numbers, but if you note things like JSON handling which is compiler sensitive, you'll note that the difference there was the biggest.

Rochus 2y ago

The post is good enough, using recognized methods and allowing reproducability. It considers different CoreCLR and Mono versions, and there is no reason to assume big performance differences since. The raw data is referenced, you can use a different scale if you prefer, but the information is the same. If you are interested, take a look at the Are-we-fast-yet benchmark suite to see that it is well defined, not just a bunch of random micro benchmarks, but also includes a few larger programs (e.g. to exercise the GC). And again: this is about the CLR, not the framework. I'm not interested in the .NET framework implementation. The build works well on all platforms without any dependencies and without downloading and compiling several gigs of stuff.

neonsunset 2y ago
I am talking about CoreCLR, which has nothing to do with .NET Framework. It has tangentially something to do with .NET's standard library but its interaction with it is the same as the one Mono has. It just executes CIL assemblies, maybe AOT compiles them if you ask build system for it. The statement "there is no reason to assume big performance differences since" is naive and ignores the performance work that went into releases 7 and 8, particularly DynamicPGO which enables the whole class of new optimizations. It also ignores the steady, significant improvements in versions 3.1, 5 and 6. These are annual jumps in performance on code that had otherwise no changes. I caution other readers to take extra grain of salt and perform their own benchmarking if they do not believe my statements. Luckily, the numbers will be very easy to interpret with difference occasionally being as much as an order of magnitude in favour of CoreCLR. Mono is easier to embed which justifies select scenarios of its usage, but if there is no such specific requirement, then you should use CoreCLR. Doing otherwise means harming your project and its users in the desire to be stubborn.
- Rochus 2y ago
  If you want to convice me, try running the referenced code on the most recent Mono and CoreCLR version and relevant architectures, and collect/report the data in an accessible way. I'm only interested in reproducible measurements, not in opinions. > Luckily, the numbers will be very easy to interpret with difference occasionally being as much as an order of magnitude in favour of CoreCLR. ... Thanks for the sales pitch.
  - neonsunset 2y ago
    It was interesting to look at how .NET deals with arbitrary assemblies from something that isn't C# or F#. In the end, I did not bother with manually linking .dll files for ILC so there will be only JIT results. In general, I do not think the CIL back-end for Oberon produces good code. It is compiler-unfriendly, ignores functions provided by .NET's CoreLib (available both under Mono and CoreCLR) to use its own poorly performing variants of them, makes everything virtual and etc. It does silly things like makes all references to a string literal into an array allocation ("test".ToCharArray()) to just do a comparison, then also treats such arrays as zero-terminated pointers despite, well, .NET arrays having length in them included already, and more. Overall, a disregard what CIL, CLI and CTS are and what they offer. What are you using .NET for if not for writing truly portable C-like? There are already pointers and manual memory management, the lowering strategy could have been much closer, and a lot of suffering is very much self-imposed. And the benchmark initially ran for so little time that it would give advantage to pure interpreter based runtime as long as it did very little, or natively compiled binaries. Surely your average program runs more than 200ms? Anyways, you can find these and a couple other notes with data and charts here: https://gist.github.com/neon-sunset/900bd310dc53dd88cbeddb5193d9104d#file-oberon-md https://gist.github.com/neon-sunset/900bd310dc53dd88cbeddb51... Geometric Mean (Mono): 1612026.37 microseconds Geometric Mean (.NET): 847482.65 microseconds Relative .NET advantage: 190.21% I did not have time to look closer in benchmarks with 1:1 numbers, but if you note things like JSON handling which is compiler sensitive, you'll note that the difference there was the biggest.