3 ms·

Count to 1B in Ruby, Python, R, JavaScript and C++

4 points by nomilk 3y ago

nomilk 3y ago
Results: c++ 1.129 seconds python 41.674 seconds ruby 8.730 seconds R 13.602 seconds js (node) 0.481 seconds What surprises me: Python slower than R and ruby. js faster than c++.
- Someone 3y ago
  They compile the “C++” (which isn’t C++ but some implementation of C++, likely Apple clang 15.0.0) without any optimizations. g++ -std=c++14 1bn.cpp time ./a.out 1000000000 When I run this on a Macbook Air M2 without optimizations versus with “-O3”, wall clock running time decreases from about 700ms to 6 ms (sometimes less than that) (Also, with “-O3”, changing the loop limit doesn’t affect running time, so clang optimizes away that loop)
- Qem 3y ago
  > What surprises me: Python slower than R and ruby. js faster than c++. Nowadays Ruby includes a JIT compiler.
andy99 3y ago
Some compilers should be able to optimize the loop away and just display the final value when the code runs. Edit: discussed previously https://news.ycombinator.com/item?id=35398298 https://news.ycombinator.com/item?id=35398298
- nomilk 3y ago
  Great find! I think the loop is essential otherwise it's an unfair comparison. But TIL that compilers can do that, so it's very interesting. I'm now wondering if there's a way to force compilers not to do that (so as to make for fair performance comparisons - e.g. it sounds like rust may do the calculation at compile time and simply print a constant at run time).
  - Someone 3y ago
    The first thing to do is pass the iteration count as an argument. That makes it less likely that a compiler optimizes this, but doesn’t guarantee it won’t do it (it is fairly easy to prove how often the loop runs and what the value of n is afterwards) Edit: for getting an idea about what compilers can do, read https://kristerw.blogspot.com/2019/04/how-llvm-optimizes-geometric-sums.html https://kristerw.blogspot.com/2019/04/how-llvm-optimizes-geo... for examples of what LLVM (the backend of various language implementations, including clang) does.
    - andy99 3y ago
      Also, what is he really trying to benchmark? Would it be informative to generate 10^9 random numbers and add them up, or is there a specific goal of finding how fast a counter is incremented?
      - nomilk 3y ago
        It’s an arbitrary benchmark, inspired by this: https://youtube.com/watch?v=VioxsWYzoJk https://youtube.com/watch?v=VioxsWYzoJk
        andy99 3y ago
        I got curious and wrote it in assembly (Disclaimer, I haven't written any assembly code since university, it was mostly chatGPT). See the attached gist. This should be the gold standard for a "fair" loop, one that's actually looping in raw assembly. https://gist.github.com/rbitr/0649972657f5d0d959eeb71a5b3e0206 https://gist.github.com/rbitr/0649972657f5d0d959eeb71a5b3e02...
        Someone 3y ago
        That removes the binary-to-decimal conversion of the result, which easily could take a significant fraction of running time when using a smart compiler.

4 points by nomilk 3y ago

Results: c++ 1.129 seconds python 41.674 seconds ruby 8.730 seconds R 13.602 seconds js (node) 0.481 seconds What surprises me: Python slower than R and ruby. js faster than c++.

Someone 3y ago
They compile the “C++” (which isn’t C++ but some implementation of C++, likely Apple clang 15.0.0) without any optimizations. g++ -std=c++14 1bn.cpp time ./a.out 1000000000 When I run this on a Macbook Air M2 without optimizations versus with “-O3”, wall clock running time decreases from about 700ms to 6 ms (sometimes less than that) (Also, with “-O3”, changing the loop limit doesn’t affect running time, so clang optimizes away that loop)
Qem 3y ago
> What surprises me: Python slower than R and ruby. js faster than c++. Nowadays Ruby includes a JIT compiler.

andy99 3y ago

Some compilers should be able to optimize the loop away and just display the final value when the code runs. Edit: discussed previously https://news.ycombinator.com/item?id=35398298 https://news.ycombinator.com/item?id=35398298

nomilk 3y ago
Great find! I think the loop is essential otherwise it's an unfair comparison. But TIL that compilers can do that, so it's very interesting. I'm now wondering if there's a way to force compilers not to do that (so as to make for fair performance comparisons - e.g. it sounds like rust may do the calculation at compile time and simply print a constant at run time).
- Someone 3y ago
  The first thing to do is pass the iteration count as an argument. That makes it less likely that a compiler optimizes this, but doesn’t guarantee it won’t do it (it is fairly easy to prove how often the loop runs and what the value of n is afterwards) Edit: for getting an idea about what compilers can do, read https://kristerw.blogspot.com/2019/04/how-llvm-optimizes-geometric-sums.html https://kristerw.blogspot.com/2019/04/how-llvm-optimizes-geo... for examples of what LLVM (the backend of various language implementations, including clang) does.
  - andy99 3y ago
    Also, what is he really trying to benchmark? Would it be informative to generate 10^9 random numbers and add them up, or is there a specific goal of finding how fast a counter is incremented?
    - nomilk 3y ago
      It’s an arbitrary benchmark, inspired by this: https://youtube.com/watch?v=VioxsWYzoJk https://youtube.com/watch?v=VioxsWYzoJk
      - andy99 3y ago
        I got curious and wrote it in assembly (Disclaimer, I haven't written any assembly code since university, it was mostly chatGPT). See the attached gist. This should be the gold standard for a "fair" loop, one that's actually looping in raw assembly. https://gist.github.com/rbitr/0649972657f5d0d959eeb71a5b3e0206 https://gist.github.com/rbitr/0649972657f5d0d959eeb71a5b3e02...
        Someone 3y ago
        That removes the binary-to-decimal conversion of the result, which easily could take a significant fraction of running time when using a smart compiler.