3 ms·

I am talking about CoreCLR, which has nothing to do with .NET Framework. It has tangentially something to do with .NET's standard library but its interaction wi

I am talking about CoreCLR, which has nothing to do with .NET Framework. It has tangentially something to do with .NET's standard library but its interaction with it is the same as the one Mono has. It just executes CIL assemblies, maybe AOT compiles them if you ask build system for it. The statement "there is no reason to assume big performance differences since" is naive and ignores the performance work that went into releases 7 and 8, particularly DynamicPGO which enables the whole class of new optimizations. It also ignores the steady, significant improvements in versions 3.1, 5 and 6. These are annual jumps in performance on code that had otherwise no changes. I caution other readers to take extra grain of salt and perform their own benchmarking if they do not believe my statements. Luckily, the numbers will be very easy to interpret with difference occasionally being as much as an order of magnitude in favour of CoreCLR. Mono is easier to embed which justifies select scenarios of its usage, but if there is no such specific requirement, then you should use CoreCLR. Doing otherwise means harming your project and its users in the desire to be stubborn.

Rochus 2y ago
If you want to convice me, try running the referenced code on the most recent Mono and CoreCLR version and relevant architectures, and collect/report the data in an accessible way. I'm only interested in reproducible measurements, not in opinions. > Luckily, the numbers will be very easy to interpret with difference occasionally being as much as an order of magnitude in favour of CoreCLR. ... Thanks for the sales pitch.
- neonsunset 2y ago
  It was interesting to look at how .NET deals with arbitrary assemblies from something that isn't C# or F#. In the end, I did not bother with manually linking .dll files for ILC so there will be only JIT results. In general, I do not think the CIL back-end for Oberon produces good code. It is compiler-unfriendly, ignores functions provided by .NET's CoreLib (available both under Mono and CoreCLR) to use its own poorly performing variants of them, makes everything virtual and etc. It does silly things like makes all references to a string literal into an array allocation ("test".ToCharArray()) to just do a comparison, then also treats such arrays as zero-terminated pointers despite, well, .NET arrays having length in them included already, and more. Overall, a disregard what CIL, CLI and CTS are and what they offer. What are you using .NET for if not for writing truly portable C-like? There are already pointers and manual memory management, the lowering strategy could have been much closer, and a lot of suffering is very much self-imposed. And the benchmark initially ran for so little time that it would give advantage to pure interpreter based runtime as long as it did very little, or natively compiled binaries. Surely your average program runs more than 200ms? Anyways, you can find these and a couple other notes with data and charts here: https://gist.github.com/neon-sunset/900bd310dc53dd88cbeddb5193d9104d#file-oberon-md https://gist.github.com/neon-sunset/900bd310dc53dd88cbeddb51... Geometric Mean (Mono): 1612026.37 microseconds Geometric Mean (.NET): 847482.65 microseconds Relative .NET advantage: 190.21% I did not have time to look closer in benchmarks with 1:1 numbers, but if you note things like JSON handling which is compiler sensitive, you'll note that the difference there was the biggest.
  - Rochus 2y ago
    Apart from the unnecessarily arrogant remarks, which by the way are completely irrelevant since the same code runs on both measured VMs, no matter how you find it, I find the result interesting and thank you very much for your effort. I had never run CoreCLR on Apple M1 before. Apparently there was indeed a further improvement on this architecture, although still the overall difference is not that dramatic. If I analyze your numbers the same way I did before, I get a geometric mean factor of 1.8, which would almost make it worth switching to CoreCLR, at least on Apple M1. But there are other critical requirements, e.g. that CoreCLR does not support all the platforms I need, and that the debugging support, as far as comparable at all, requires a completely different integration. Just for the speed-up on Apple it's not worth switching. As soon as I find some time, I will try again on x86 and x86_64 and see how it looks there in comparison to my 2021 measurements.
    - Rochus 2y ago
      Here are some comparisons to my 2021 measurements. - Mono 5 and Mono 6 on Windows 10 x86 achieved about the same performance (factor 1.04) - Taking Mono 5 as a reference on Windows 10 x64, CoreCLR 3 had about the same performance, CoreCLR 5 was factor 1.3 faster than Mono 5, and CoreCLR 6 was factor 1.2 faster than Mono 5. - The measurements done by neonsunset on Mac M1 show that CoreCLR 9 is factor 1.8 faster than Mono 6; therefore CoreCLR 9 (M1) seems to be around factor 1.4 faster than CoreCLR 5 (x64).
      - neonsunset 2y ago
        The issue of poor lowering strategy remains. The back-end should be using structs, pointers and manual memory management if this is what it does with C. C# supports full set of features to make it possible including (now) raw function pointers. Just no C macros, sorry. It should not even be close to Mono in performance. It should be close to C, given correct back-end output, which it isn't. At this point, outputting C# would have produced much better results that do not defeat compiler optimizations, where CoreCLR performs rather well despite the CIL quality, not because of it. And even then, you can easily see that either computationally intensive or logically complex benchmarks show the biggest difference as they stress out compiler capability the most. In any case, the option to target C# instead would result in guaranteed correct CIL form, with loops written in a way that does not occasionally break loop recognition, does not introduce unnecessary memory and abstraction overhead and does not have the issues with the basics like that one with string comparison that allocates each time for no reason. Also note that compiler improvements are generic - most affect any platform supported, with platform-specific peephole optimizations responsible for cleaning up the codegen and eking out the last %. If-conversion pass for ARM64 is going to be an exception, but on the other hand x86_64 offers wider vectors which .NET makes use of.
kragen 2y ago
i notice that debian still doesn't have a .net core package. do you know what the obstacle is? is there a hidden licensing problem, or are the debian developers having trouble getting the released source to compile successfully? they do have a mono package so it can't be that they're morally opposed to c#
- neonsunset 2y ago
  It should be available depending on your source list? sudo apt install dotnet-sdk-8.0
  - kragen 2y ago
    https://packages.debian.org/search?suite=all&searchon=names&keywords=dotnet-sdk https://packages.debian.org/search?suite=all&searchon=names&... says, 'your search gave no results', showing that nothing similar to dotnet-sdk-8.0 exists in debian. what i'm asking is why not: is it a licensing problem or a difficulty in building? i'm not interested in whether some fly-by-night third-party software provider offers a repository of debs
    - neonsunset 2y ago
      Most distro feeds do have it, hell even on FreeBSD it's just `pkg install lang/dotnet`. I guess Debian is special just like that (not that I care), there can't be a licensing issue because it's all MIT and everything can be built (takes a lot of time though) from a monorepo https://github.com/dotnet/dotnet https://github.com/dotnet/dotnet. In any case, there's an official guide: https://learn.microsoft.com/en-us/dotnet/core/install/linux-debian https://learn.microsoft.com/en-us/dotnet/core/install/linux-...
      - kragen 2y ago
        thanks! yeah, i'd checked out the official guide previously, but it doesn't explain how to build from source, just how to download prebuilt binaries from some fly-by-night third-party software vendor website. that's part of what made me suspicious!

Rochus 2y ago

If you want to convice me, try running the referenced code on the most recent Mono and CoreCLR version and relevant architectures, and collect/report the data in an accessible way. I'm only interested in reproducible measurements, not in opinions. > Luckily, the numbers will be very easy to interpret with difference occasionally being as much as an order of magnitude in favour of CoreCLR. ... Thanks for the sales pitch.

neonsunset 2y ago
It was interesting to look at how .NET deals with arbitrary assemblies from something that isn't C# or F#. In the end, I did not bother with manually linking .dll files for ILC so there will be only JIT results. In general, I do not think the CIL back-end for Oberon produces good code. It is compiler-unfriendly, ignores functions provided by .NET's CoreLib (available both under Mono and CoreCLR) to use its own poorly performing variants of them, makes everything virtual and etc. It does silly things like makes all references to a string literal into an array allocation ("test".ToCharArray()) to just do a comparison, then also treats such arrays as zero-terminated pointers despite, well, .NET arrays having length in them included already, and more. Overall, a disregard what CIL, CLI and CTS are and what they offer. What are you using .NET for if not for writing truly portable C-like? There are already pointers and manual memory management, the lowering strategy could have been much closer, and a lot of suffering is very much self-imposed. And the benchmark initially ran for so little time that it would give advantage to pure interpreter based runtime as long as it did very little, or natively compiled binaries. Surely your average program runs more than 200ms? Anyways, you can find these and a couple other notes with data and charts here: https://gist.github.com/neon-sunset/900bd310dc53dd88cbeddb5193d9104d#file-oberon-md https://gist.github.com/neon-sunset/900bd310dc53dd88cbeddb51... Geometric Mean (Mono): 1612026.37 microseconds Geometric Mean (.NET): 847482.65 microseconds Relative .NET advantage: 190.21% I did not have time to look closer in benchmarks with 1:1 numbers, but if you note things like JSON handling which is compiler sensitive, you'll note that the difference there was the biggest.
- Rochus 2y ago
  Apart from the unnecessarily arrogant remarks, which by the way are completely irrelevant since the same code runs on both measured VMs, no matter how you find it, I find the result interesting and thank you very much for your effort. I had never run CoreCLR on Apple M1 before. Apparently there was indeed a further improvement on this architecture, although still the overall difference is not that dramatic. If I analyze your numbers the same way I did before, I get a geometric mean factor of 1.8, which would almost make it worth switching to CoreCLR, at least on Apple M1. But there are other critical requirements, e.g. that CoreCLR does not support all the platforms I need, and that the debugging support, as far as comparable at all, requires a completely different integration. Just for the speed-up on Apple it's not worth switching. As soon as I find some time, I will try again on x86 and x86_64 and see how it looks there in comparison to my 2021 measurements.
  - Rochus 2y ago
    Here are some comparisons to my 2021 measurements. - Mono 5 and Mono 6 on Windows 10 x86 achieved about the same performance (factor 1.04) - Taking Mono 5 as a reference on Windows 10 x64, CoreCLR 3 had about the same performance, CoreCLR 5 was factor 1.3 faster than Mono 5, and CoreCLR 6 was factor 1.2 faster than Mono 5. - The measurements done by neonsunset on Mac M1 show that CoreCLR 9 is factor 1.8 faster than Mono 6; therefore CoreCLR 9 (M1) seems to be around factor 1.4 faster than CoreCLR 5 (x64).
    - neonsunset 2y ago
      The issue of poor lowering strategy remains. The back-end should be using structs, pointers and manual memory management if this is what it does with C. C# supports full set of features to make it possible including (now) raw function pointers. Just no C macros, sorry. It should not even be close to Mono in performance. It should be close to C, given correct back-end output, which it isn't. At this point, outputting C# would have produced much better results that do not defeat compiler optimizations, where CoreCLR performs rather well despite the CIL quality, not because of it. And even then, you can easily see that either computationally intensive or logically complex benchmarks show the biggest difference as they stress out compiler capability the most. In any case, the option to target C# instead would result in guaranteed correct CIL form, with loops written in a way that does not occasionally break loop recognition, does not introduce unnecessary memory and abstraction overhead and does not have the issues with the basics like that one with string comparison that allocates each time for no reason. Also note that compiler improvements are generic - most affect any platform supported, with platform-specific peephole optimizations responsible for cleaning up the codegen and eking out the last %. If-conversion pass for ARM64 is going to be an exception, but on the other hand x86_64 offers wider vectors which .NET makes use of.

kragen 2y ago

i notice that debian still doesn't have a .net core package. do you know what the obstacle is? is there a hidden licensing problem, or are the debian developers having trouble getting the released source to compile successfully? they do have a mono package so it can't be that they're morally opposed to c#

neonsunset 2y ago
It should be available depending on your source list? sudo apt install dotnet-sdk-8.0
- kragen 2y ago
  https://packages.debian.org/search?suite=all&searchon=names&keywords=dotnet-sdk https://packages.debian.org/search?suite=all&searchon=names&... says, 'your search gave no results', showing that nothing similar to dotnet-sdk-8.0 exists in debian. what i'm asking is why not: is it a licensing problem or a difficulty in building? i'm not interested in whether some fly-by-night third-party software provider offers a repository of debs
  - neonsunset 2y ago
    Most distro feeds do have it, hell even on FreeBSD it's just `pkg install lang/dotnet`. I guess Debian is special just like that (not that I care), there can't be a licensing issue because it's all MIT and everything can be built (takes a lot of time though) from a monorepo https://github.com/dotnet/dotnet https://github.com/dotnet/dotnet. In any case, there's an official guide: https://learn.microsoft.com/en-us/dotnet/core/install/linux-debian https://learn.microsoft.com/en-us/dotnet/core/install/linux-...
    - kragen 2y ago
      thanks! yeah, i'd checked out the official guide previously, but it doesn't explain how to build from source, just how to download prebuilt binaries from some fly-by-night third-party software vendor website. that's part of what made me suspicious!