4 ms·

I've been rewriting Python->C for nearly 20 years now. The expected speedup is around 100x, or 1000x for numerical stuff or allocation-heavy work that can be d

by klyrs 3y ago

I've been rewriting Python->C for nearly 20 years now. The expected speedup is around 100x, or 1000x for numerical stuff or allocation-heavy work that can be done statically. Whenever you get 10,000x or above, it's because you've written a better algorithm. You can't generalize that. A 35k speedup is a cool demo but should be regarded as hype.

nomel 3y ago
> The expected speedup is around 100x, or 1000x for numerical stuff What if you stay in the realm of numpy? What's the biggest offender that you see?
- klyrs 3y ago
  > What if you stay in the realm of numpy? You mean, what if you're only doing matrix stuff? Then it's probably easier to let numpy do the heavy lifting. You'll probably take less than a 5x performance hit, if you're doing numpy right. And if you're doing matrix multiplication, numpy will end up faster because it's backed by a BLAS, which mortals such as myself know better than to compete with. > What's the biggest offender that you see? Umm... every line of Python? Member access. Function calls. Dictionaries that can fundamentally be mapped to int-indexed arrays. Reference counting. Tuple allocation. One fun exercise is to take your vanilla python code, compile it in Cython with the -a flag to produce an HTML annotation. Click on the yellowest lines, and it shows you the gory details of what Cython does to emulate CPython. It's not exactly what CPython is doing (for example, Cython elides the virtual machine), but it's close enough to see where time is spent. Put the same code through the python disassembler "dis" to see what virtual machine operations are emitted, and paw through the main evaluation loop [1]; or take a guided walkthrough at [2]. [1] https://github.com/python/cpython/blob/v3.6.14/Python/ceval.c#L1274 https://github.com/python/cpython/blob/v3.6.14/Python/ceval.... (note this is an old version, you can change that in the url) [2] https://leanpub.com/insidethepythonvirtualmachine/read https://leanpub.com/insidethepythonvirtualmachine/read
  - vbarrielle 3y ago
    Due to the possibility to fuse multiple operations in C++ (whereas you often have intermediate arrays in numpy), I routinely get 20x speedups when porting from numpy to C++. Good libraries like eigen help a lot.
    - arjvik 3y ago
      What's an example of fusing operations? Are you talking about combinations of operations that are used commonly enough to warrant Eigen methods that perform them at once in SIMD?
      - celrod 3y ago
        Probably that eigen uses expression templates to avoid the needles creation of temporaries.
      - cozzyd 3y ago
        most non-trivial numpy operations require temporaries that require new allocations and copies. Eigen3's design lets you avoid these through clever compilation tricks while remaining high-level. sometimes numpy can elide those (e.g. why a+=b is faster than a=a+b) but this it not possible in general. Sometimes people use monstrosities like einsum... but I find it more intuitive to just write in C or C++... In addition to the time spent in allocation / gc / needless copying, the memory footprint can be higher by a factor of a few (or more...).
        klyrs 3y ago
        Yep, einsum is included in "doing numpy right." And for what it's worth, it's horrid to use and still won't get around cases like x -> cos(x). I haven't needed the power of eigen for a couple of years, but I appreciate the tip.
  - m3affan 3y ago
    This guy cythons
- eslaught 3y ago
  You can get on the order of 10-30x speedup over NumPy by reducing the allocation of temporaries and fusing across operations. See: Weld: A Common Runtime for High Performance Data Analytics https://dspace.mit.edu/bitstream/handle/1721.1/137425/cidr_weld.pdf?sequence=2&isAllowed=y https://dspace.mit.edu/bitstream/handle/1721.1/137425/cidr_w...
kouteiheika 3y ago
> The expected speedup is around 100x, or 1000x for numerical stuff or allocation-heavy work that can be done statically. Whenever you get 10,000x or above, it's because you've written a better algorithm. Anecdotally I recently rewrote a piece of Python code in Rust and got ~300x speedup, but let's be conservative and give it 100x. Now let's extrapolate from that. In native code you can use SIMD, and that can give you a 10x speedup, so now we're at 1000x. In native code you can also easily use multiple threads, so assuming a machine with a reasonably high number of cores, let's say 32 of them (because that's what I had for the last 4 years), we're now at 32000x speedup. So to me those are very realistic numbers, but of course assuming the problem you're solving can be sped up with SIMD and multiple threads, which is not always the case. So you're probably mostly right.
- piyh 3y ago
  Python can use multiprocessing with a shared nothing architecture to use those 32 threads.
  - malodyets 3y ago
    I was about to say the same thing. Multiprocessing on Python works great and isn’t even very hard if you use say async_apply with a Pool. Comparing single-threaded Python with multiprocesssing in Language X is unfair if not disingenuous.
    - mlyle 3y ago
      > Multiprocessing on Python works great and isn’t even very hard if you use say async_apply with a Pool. Multiprocessing works great if you don't really need a shared memory space for your task. If it's very loosely coupled, that's fine. But if you use something that can benefit from real threading, Python clamps you to about 1.5-2.5 cores worth of throughput very often.
    - heavyset_go 3y ago
      There's a serialization overhead both on dispatch and return that makes multiprocessing in Python unsuitable for some problems that would otherwise be solved well with threads in other languages.
sk0g 3y ago
I wrote a simple Monte Carlo implementation in Python 3.11 and Rust. Python managed 10 million checks in a certain timeframe, while Rust was could perform 9 billion checks in the same timeframe. That's about a 900x speedup, if I'm not mistaken. I suspect Mojo's advertised speedup is through the same process, except on benchmarks that are not dominated by syscalls (RNG calls in this instance). The one difference was the Rust one used a parallel iterator (rayon + one liner change), whereas I have found Python to be more pain than it's worth, for most usecases.
- deleted 3y ago
  [deleted]
3836293648 3y ago
Iirc, the 35kx number included parallelisation
- nerpderp82 3y ago
  Because they are targeting the GPU, write a GPGPU shader and call it from Python and you will get the same number. Or use Jax or Taichi.

by klyrs 3y ago

nomel 3y ago

> The expected speedup is around 100x, or 1000x for numerical stuff What if you stay in the realm of numpy? What's the biggest offender that you see?

klyrs 3y ago
> What if you stay in the realm of numpy? You mean, what if you're only doing matrix stuff? Then it's probably easier to let numpy do the heavy lifting. You'll probably take less than a 5x performance hit, if you're doing numpy right. And if you're doing matrix multiplication, numpy will end up faster because it's backed by a BLAS, which mortals such as myself know better than to compete with. > What's the biggest offender that you see? Umm... every line of Python? Member access. Function calls. Dictionaries that can fundamentally be mapped to int-indexed arrays. Reference counting. Tuple allocation. One fun exercise is to take your vanilla python code, compile it in Cython with the -a flag to produce an HTML annotation. Click on the yellowest lines, and it shows you the gory details of what Cython does to emulate CPython. It's not exactly what CPython is doing (for example, Cython elides the virtual machine), but it's close enough to see where time is spent. Put the same code through the python disassembler "dis" to see what virtual machine operations are emitted, and paw through the main evaluation loop [1]; or take a guided walkthrough at [2]. [1] https://github.com/python/cpython/blob/v3.6.14/Python/ceval.c#L1274 https://github.com/python/cpython/blob/v3.6.14/Python/ceval.... (note this is an old version, you can change that in the url) [2] https://leanpub.com/insidethepythonvirtualmachine/read https://leanpub.com/insidethepythonvirtualmachine/read
- vbarrielle 3y ago
  Due to the possibility to fuse multiple operations in C++ (whereas you often have intermediate arrays in numpy), I routinely get 20x speedups when porting from numpy to C++. Good libraries like eigen help a lot.
  - arjvik 3y ago
    What's an example of fusing operations? Are you talking about combinations of operations that are used commonly enough to warrant Eigen methods that perform them at once in SIMD?
    - celrod 3y ago
      Probably that eigen uses expression templates to avoid the needles creation of temporaries.
    - cozzyd 3y ago
      most non-trivial numpy operations require temporaries that require new allocations and copies. Eigen3's design lets you avoid these through clever compilation tricks while remaining high-level. sometimes numpy can elide those (e.g. why a+=b is faster than a=a+b) but this it not possible in general. Sometimes people use monstrosities like einsum... but I find it more intuitive to just write in C or C++... In addition to the time spent in allocation / gc / needless copying, the memory footprint can be higher by a factor of a few (or more...).
      - klyrs 3y ago
        Yep, einsum is included in "doing numpy right." And for what it's worth, it's horrid to use and still won't get around cases like x -> cos(x). I haven't needed the power of eigen for a couple of years, but I appreciate the tip.
- m3affan 3y ago
  This guy cythons
eslaught 3y ago
You can get on the order of 10-30x speedup over NumPy by reducing the allocation of temporaries and fusing across operations. See: Weld: A Common Runtime for High Performance Data Analytics https://dspace.mit.edu/bitstream/handle/1721.1/137425/cidr_weld.pdf?sequence=2&isAllowed=y https://dspace.mit.edu/bitstream/handle/1721.1/137425/cidr_w...

kouteiheika 3y ago

> The expected speedup is around 100x, or 1000x for numerical stuff or allocation-heavy work that can be done statically. Whenever you get 10,000x or above, it's because you've written a better algorithm. Anecdotally I recently rewrote a piece of Python code in Rust and got ~300x speedup, but let's be conservative and give it 100x. Now let's extrapolate from that. In native code you can use SIMD, and that can give you a 10x speedup, so now we're at 1000x. In native code you can also easily use multiple threads, so assuming a machine with a reasonably high number of cores, let's say 32 of them (because that's what I had for the last 4 years), we're now at 32000x speedup. So to me those are very realistic numbers, but of course assuming the problem you're solving can be sped up with SIMD and multiple threads, which is not always the case. So you're probably mostly right.

piyh 3y ago
Python can use multiprocessing with a shared nothing architecture to use those 32 threads.
- malodyets 3y ago
  I was about to say the same thing. Multiprocessing on Python works great and isn’t even very hard if you use say async_apply with a Pool. Comparing single-threaded Python with multiprocesssing in Language X is unfair if not disingenuous.
  - mlyle 3y ago
    > Multiprocessing on Python works great and isn’t even very hard if you use say async_apply with a Pool. Multiprocessing works great if you don't really need a shared memory space for your task. If it's very loosely coupled, that's fine. But if you use something that can benefit from real threading, Python clamps you to about 1.5-2.5 cores worth of throughput very often.
  - heavyset_go 3y ago
    There's a serialization overhead both on dispatch and return that makes multiprocessing in Python unsuitable for some problems that would otherwise be solved well with threads in other languages.

sk0g 3y ago

I wrote a simple Monte Carlo implementation in Python 3.11 and Rust. Python managed 10 million checks in a certain timeframe, while Rust was could perform 9 billion checks in the same timeframe. That's about a 900x speedup, if I'm not mistaken. I suspect Mojo's advertised speedup is through the same process, except on benchmarks that are not dominated by syscalls (RNG calls in this instance). The one difference was the Rust one used a parallel iterator (rayon + one liner change), whereas I have found Python to be more pain than it's worth, for most usecases.

deleted 3y ago
[deleted]

3836293648 3y ago

Iirc, the 35kx number included parallelisation

nerpderp82 3y ago
Because they are targeting the GPU, write a GPGPU shader and call it from Python and you will get the same number. Or use Jax or Taichi.