5 ms·

So go on then - let's hear how fast rust can do it? Based on the trends of "<something something> so we rewrote our entire production stack in rust!" I am surp

So go on then - let's hear how fast rust can do it? Based on the trends of "<something something> so we rewrote our entire production stack in rust!" I am surprised there is not already a comment here! I'd also be interested in how fast JavaScript/V8 executes by comparison.

jabart 3y ago
The .NET community started their own version and included some other languages. I think I saw Rust at 1.4s or so but was disqualified because it was optimized to that dataset (no idea how they determined that). Someone wrote some SIMD code for c# and was like 1s. Shaving some milliseconds between c#, c++, and rust in the final end for 1 billion rows is still impressive.
- anonymoushn 3y ago
  I think you're talking about a Rust solution that works by reading a small prefix of the input and generating a perfect hash with no handling for misses. So it will be wrong for any input that contains station names that do not occur in that prefix.
- buybackoff 3y ago
  I maintain cross-language results. There will be a huge update soon, including AWS metal instances running on many combinations of cores/HT-threads. What you mention from Rust is this: https://github.com/RagnarGrootKoerkamp/1brc/ https://github.com/RagnarGrootKoerkamp/1brc/ The author says himself that his solution is not compliant in the readme. So far the top place is not clear. .NET with noahfalk, C/C++ from lehuyduc, dzaima and austindonisan require more tests. The last looks the fastest so far on the default dataset, but has issues. Here is my blog post with details: https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-fastest-on-linux-my-optimization-journey/# https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-...
- za3faran 3y ago
  Do you have a link to the .NET community work?
  - banashark 3y ago
    https://github.com/praeclarum/1brc https://github.com/praeclarum/1brc
    - pron 3y ago
      That's on a machine with more cores. The "baseline" 04:49.679 Java solution runs in 03:17.70 on that machine.
- DaiPlusPlus 3y ago
  > Someone wrote some SIMD code for c# and was like 1s. CLR performance can be considered nondeterministic given it JITs and performs profile-guided-optimizations - unless you go with AOT, but there's still going to be an initialization delay, and differences in cold vs. warm. vs hot-start scenarios. In order to be fair to all languages/platforms/runtimes involved, how is that managed?
  - neonsunset 3y ago
    The answers to these questions are at the links posted in a sibling comment a couple of hours ago.
  - radicalbyte 3y ago
    ..which we then proceed to run in a container running on a VM on a random machine in the cloud :-)
hyfgfh 3y ago
It depends on the machine, but some people got very fast results. https://github.com/gunnarmorling/1brc/discussions/57 https://github.com/gunnarmorling/1brc/discussions/57 Why would you use Javascript for that?
- pulse7 3y ago
  Some people believe JavaScript/V8 is faster than Java so I would also be glad to see the comparison here...
ThePhysicist 3y ago
There's a Reddit thread about that [1], though I'm not sure how the performance numbers can be compared as the hardware is probably different. 1: https://www.reddit.com/r/rust/comments/18ws370/optimizing_a_one_billion_row_challenge_in_rust/ https://www.reddit.com/r/rust/comments/18ws370/optimizing_a_...
Dowwie 3y ago
Search "1brc rust" on Twitter and you can see people hard at work competing on optimized Rust solutions

jabart 3y ago

The .NET community started their own version and included some other languages. I think I saw Rust at 1.4s or so but was disqualified because it was optimized to that dataset (no idea how they determined that). Someone wrote some SIMD code for c# and was like 1s. Shaving some milliseconds between c#, c++, and rust in the final end for 1 billion rows is still impressive.

anonymoushn 3y ago
I think you're talking about a Rust solution that works by reading a small prefix of the input and generating a perfect hash with no handling for misses. So it will be wrong for any input that contains station names that do not occur in that prefix.
buybackoff 3y ago
I maintain cross-language results. There will be a huge update soon, including AWS metal instances running on many combinations of cores/HT-threads. What you mention from Rust is this: https://github.com/RagnarGrootKoerkamp/1brc/ https://github.com/RagnarGrootKoerkamp/1brc/ The author says himself that his solution is not compliant in the readme. So far the top place is not clear. .NET with noahfalk, C/C++ from lehuyduc, dzaima and austindonisan require more tests. The last looks the fastest so far on the default dataset, but has issues. Here is my blog post with details: https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-fastest-on-linux-my-optimization-journey/# https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-...
za3faran 3y ago
Do you have a link to the .NET community work?
- banashark 3y ago
  https://github.com/praeclarum/1brc https://github.com/praeclarum/1brc
  - pron 3y ago
    That's on a machine with more cores. The "baseline" 04:49.679 Java solution runs in 03:17.70 on that machine.
DaiPlusPlus 3y ago
> Someone wrote some SIMD code for c# and was like 1s. CLR performance can be considered nondeterministic given it JITs and performs profile-guided-optimizations - unless you go with AOT, but there's still going to be an initialization delay, and differences in cold vs. warm. vs hot-start scenarios. In order to be fair to all languages/platforms/runtimes involved, how is that managed?
- neonsunset 3y ago
  The answers to these questions are at the links posted in a sibling comment a couple of hours ago.
- radicalbyte 3y ago
  ..which we then proceed to run in a container running on a VM on a random machine in the cloud :-)

hyfgfh 3y ago

It depends on the machine, but some people got very fast results. https://github.com/gunnarmorling/1brc/discussions/57 https://github.com/gunnarmorling/1brc/discussions/57 Why would you use Javascript for that?

pulse7 3y ago
Some people believe JavaScript/V8 is faster than Java so I would also be glad to see the comparison here...

ThePhysicist 3y ago

There's a Reddit thread about that [1], though I'm not sure how the performance numbers can be compared as the hardware is probably different. 1: https://www.reddit.com/r/rust/comments/18ws370/optimizing_a_one_billion_row_challenge_in_rust/ https://www.reddit.com/r/rust/comments/18ws370/optimizing_a_...

Dowwie 3y ago

Search "1brc rust" on Twitter and you can see people hard at work competing on optimized Rust solutions