3 ms·
> It is irrelevant that the benchmark is written in Oberon. It is not irrelevant from my perspective. It might be mostly irrelevant if Oberon, as you say, "fai
by rndgermandude 5y ago
> It is irrelevant that the benchmark is written in Oberon.
It is not irrelevant from my perspective. It might be mostly irrelevant if Oberon, as you say, "fairly directly" translates to CRL/IL, that is, there is no need to introduce any (inefficient) glue constructs to support Oberon features. I didn't/do not know this, but if you say this to be the case now, good, that answers my doubts in that regard.
However, this still does not speak to how good the codegen of your Oberon-to-CLR/IL compiler is. Yes, the mono or CoreCLR JIT will apply further optimizations, but in a significant part it relies on the quality of the codegen of the IL it gets passed to do so.
>The fact that Mono is only a factor of two behind is pretty good.
According to your benchmarks. I'd still be a lot more interested in how well the CoreCRL would fare when handed CLR/IL generated from idiomatic C# or F# by the dotnet (Roslyn) compiler. On recent hardware. Or at least a recent hardware architecture.
Right now, it seems to me you're doing something analogous to compiling C with tcc and then making general performance statements about C performance based on tcc generated machine code. Even your own mono-3 and mono-5 results differ, so choice of runtime is clearly a factor.
>It is irrelevant that C# is the most commonly used language on .NET; it is by definition and design a "common language infrastructure", and the article comments on .NET in general (and makes claims at least as adventurous as yours, the ones discussed here also without supporting evidence).
It is not irrelevant. The claims unsupported by any evidence are yours by the way. You made statements about performance, and all you presented is what I consider at best highly questionable methodology. If you make such claims, and try to back them up by "data", you'll have to demonstrate that your Oberon+-benchmark-code-compiled-to-CLR/IR-as-run-on-old-mono-on-outdated-hardware benchmark methodology is indeed something that can be used to make such claims about ".NET" in a sound manner. As I said, I doubt that, and that's all I did, express doubts about your methodology, and why I have these doubts.
> Regarding your nonsensical claims about Span<> and stackalloc, I recommend that you take a look at ECMA-335 and its development over time (and at the generated code).
I have been following standard library and CLR/IR development, and Rosyln compiler development, and therefore know that my statements are not nonsensical. E.g. here[0] is the dotnet "Augments" document that lists, aside from many clarifications, outright additions not present in the latest ECMA-335 spec. See e.g. the changes/additions regarding Byref-like types, which in practice allow for all kinds of optimizations, a lot of them implemented in CoreCLR 5 or 6, but of course only enabled if the compiler's codegen emits these new byref-like type variants.
Furthermore, a lot of the optimizations that current-generation to-CLR/IL compilers and runtimes/JITs perform do not rely on changes to ECMA-335 at all. E.g. as I mentioned above, your mono-3 runs the CLR/IL your compiler generated significantly faster than your mono-5.
>Also your assertion about the relevance of the HDD and the other mentioned hardware aspects makes me doubt whether you have any idea at all about what you are writing here
You're running on a somewhat memory-constrained old machine, so even if you try to run your benchmarks "hot", i.e. have everything required loaded into OS-level caches before the timing starts, I wouldn't be convinced that this is necessarily the case (disk cache pages might have been evicted, and pages in general might have been swapped out and need swapping back in again during a run, and if that happens the HDD performance would play a major role). Additionally, laptops (with relatively early versions SpeedStep-capable CPUs, as your is) are particularly susceptible to additional issues that might adversely affect the reliability of benchmark results, e.g. thermal throttling.
If any of this happens, it might be (but is not necessarily) detectable in e.g. stddev (your benchmarks pdf lacks any mention of stddev, so I cannot know).
I remember trying to benchmark something compute-heavy on my HP Elitebook 8440p[2] back in the day, and gave up after I couldn't get the thing to produce any sane results for different runs of the same code.
The choice of your hardware might as well matter and create vastly different proportional results between the different runtimes you test when the same benchmarks are run on different hardware, even if there aren't any of the problems I described.
As I wrote, your choice of hardware and software (and my anecdote) does not necessarily invalidate your results, but it might render your results rather meaningless for other, more commonly used current generation hardware. And that's something to be aware of.
>so why should people take the effort of making measurements and publishing data?
Sorry, but the effort you obviously put into generating your benchmarks and data does not preclude your work from being poked-and-prodded, questioned and criticized.
[0] https://github.com/dotnet/runtime/blob/main/docs/design/specs/Ecma-335-Augments.md https://github.com/dotnet/runtime/blob/main/docs/design/spec...
[1] I am furthermore not sure if you run the same CLR/IL; the mono-3 vs mono-5 seem to be using different versions of your OBXMC as indicated by the different dates that seem to correspond to version numbers for the other things benchmarked?
[2] The variant with the Core i7-720QM if I remember correctly, slightly newer than yours, but probably even more susceptible to SpeedStep and thermal throttling.