4 ms·

The .NET community started their own version and included some other languages. I think I saw Rust at 1.4s or so but was disqualified because it was optimized t

by jabart 3y ago

The .NET community started their own version and included some other languages. I think I saw Rust at 1.4s or so but was disqualified because it was optimized to that dataset (no idea how they determined that). Someone wrote some SIMD code for c# and was like 1s. Shaving some milliseconds between c#, c++, and rust in the final end for 1 billion rows is still impressive.

anonymoushn 3y ago
I think you're talking about a Rust solution that works by reading a small prefix of the input and generating a perfect hash with no handling for misses. So it will be wrong for any input that contains station names that do not occur in that prefix.
buybackoff 3y ago
I maintain cross-language results. There will be a huge update soon, including AWS metal instances running on many combinations of cores/HT-threads. What you mention from Rust is this: https://github.com/RagnarGrootKoerkamp/1brc/ https://github.com/RagnarGrootKoerkamp/1brc/ The author says himself that his solution is not compliant in the readme. So far the top place is not clear. .NET with noahfalk, C/C++ from lehuyduc, dzaima and austindonisan require more tests. The last looks the fastest so far on the default dataset, but has issues. Here is my blog post with details: https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-fastest-on-linux-my-optimization-journey/# https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-...
za3faran 3y ago
Do you have a link to the .NET community work?
- banashark 3y ago
  https://github.com/praeclarum/1brc https://github.com/praeclarum/1brc
  - pron 3y ago
    That's on a machine with more cores. The "baseline" 04:49.679 Java solution runs in 03:17.70 on that machine.
DaiPlusPlus 3y ago
> Someone wrote some SIMD code for c# and was like 1s. CLR performance can be considered nondeterministic given it JITs and performs profile-guided-optimizations - unless you go with AOT, but there's still going to be an initialization delay, and differences in cold vs. warm. vs hot-start scenarios. In order to be fair to all languages/platforms/runtimes involved, how is that managed?
- neonsunset 3y ago
  The answers to these questions are at the links posted in a sibling comment a couple of hours ago.
- radicalbyte 3y ago
  ..which we then proceed to run in a container running on a VM on a random machine in the cloud :-)

by jabart 3y ago

anonymoushn 3y ago

I think you're talking about a Rust solution that works by reading a small prefix of the input and generating a perfect hash with no handling for misses. So it will be wrong for any input that contains station names that do not occur in that prefix.

buybackoff 3y ago

I maintain cross-language results. There will be a huge update soon, including AWS metal instances running on many combinations of cores/HT-threads. What you mention from Rust is this: https://github.com/RagnarGrootKoerkamp/1brc/ https://github.com/RagnarGrootKoerkamp/1brc/ The author says himself that his solution is not compliant in the readme. So far the top place is not clear. .NET with noahfalk, C/C++ from lehuyduc, dzaima and austindonisan require more tests. The last looks the fastest so far on the default dataset, but has issues. Here is my blog post with details: https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-fastest-on-linux-my-optimization-journey/# https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-...

za3faran 3y ago

Do you have a link to the .NET community work?

banashark 3y ago
https://github.com/praeclarum/1brc https://github.com/praeclarum/1brc
- pron 3y ago
  That's on a machine with more cores. The "baseline" 04:49.679 Java solution runs in 03:17.70 on that machine.

DaiPlusPlus 3y ago

> Someone wrote some SIMD code for c# and was like 1s. CLR performance can be considered nondeterministic given it JITs and performs profile-guided-optimizations - unless you go with AOT, but there's still going to be an initialization delay, and differences in cold vs. warm. vs hot-start scenarios. In order to be fair to all languages/platforms/runtimes involved, how is that managed?

neonsunset 3y ago
The answers to these questions are at the links posted in a sibling comment a couple of hours ago.
radicalbyte 3y ago
..which we then proceed to run in a container running on a VM on a random machine in the cloud :-)