6 ms·

Analyzing Data 170,000x Faster with Python

ndr 3y ago
For context, this is in response to 'Analyzing Data 180,000x Faster with Rust' [0], previously discussed on HN [1] [0] https://willcrichton.net/notes/k-corrset/ https://willcrichton.net/notes/k-corrset/ [1] https://news.ycombinator.com/item?id=37964161 https://news.ycombinator.com/item?id=37964161
orf 3y ago
I’m all for not prematurely optimising and the point of the authors post is still valid, but it is hard to argue that the resulting code is more readable or understandable than the rust code. The rust code is pretty simple, and plain, and doesn’t need any of these tricks
- sgarland 3y ago
  I don’t think Numpy is complex for anyone doing data analysis. Numba is effectively transparent, since it’s just a decorator. It’s very readable to me (not a data scientist, but I’m decent at Python), YMMV.
- datadrivenangel 3y ago
  There are benefits to standardizing on a single language and codebase, so it may be worth having some code that is less readable or performant in order to optimize for the global maximum and not prematurely introduce too many languages.
  - appplication 3y ago
    My least favorite thing about performance optimizations is how hard they are to guard against regressions with standard CI/CD. Unit tests, integration tests, etc. are rarely the right tools. And in some systems (e.g. distributed data processing), local performance often does not indicate true performance at scale.
    - sgarland 3y ago
      If you want truly accurate results, you have to have a dedicated machine – not a VM (unless you’re controlling the hypervisor, and can guarantee no overcommit), not a container, but an entire machine – and tightly control every aspect of it. Updated packages? All results are invalid until you can prove the baseline hasn’t changed. I don’t think most people need this level of accuracy, but it is something to consider. Database queries are even worse, because not only do you need representative data at a similar scale (or you’ve scaled both hardware and working set accurately), but you have to hope that the table/column stats in prod and test are kept at a similar level of accuracy, which is more art than science. Did the query regress because of the change, or because writes to an unrelated table have increased load and the analysis daemon is now struggling? Did it regress in test because someone else pushed a change to a rarely-used query, causing buffer churn when CI ran your job? Etc.
      - appplication 3y ago
        I agree with all of this. Because of this, even at massive scale it’s hard to justify spending the time and effort needed to get this level of testing.
  - FridgeSeal 3y ago
    Conversely, freeing data science/numerical analysis tooling from the tire-fire of the Python ecosystem might actually be a net benefit for the community at large. Get a bit more variety into the options, explore some alternative approaches. Who knows, maybe some ideas from other programming languages will yield some useful advantages.
    - sgarland 3y ago
      Python’s ecosystem is great IFF you use 3rd party tooling carefully, and understand how and when to use it. The biggest one is to use venvs liberally, and to let as few things as possible interact with system Python. If there’s a CLI tool written in Python, ideally install it with pipx. Use Poetry for every project. Unfortunately these aren’t inherently obvious until you have a raging tire fire, and even then, you have to know the source of it and that better options exist. But hey, at least it isn’t Node.
      - pjmlp 3y ago
        Node has a JIT by default, while Python is still fighting to get one that is widely accepted. I still have hopes that Julia and Ruby competition makes it change.
        sgarland 3y ago
        In theory, Node is faster than Python due to this, among other things. In practice, I have seen Node apps blocking or crashing far more often than Python apps, which drags the performance down just a bit. In the meantime, @njit decorator on Python functions, and boom, done.
        pjmlp 3y ago
        I am yet to see such thing, to each its own I guess. @njit doesn't work in regular Python, requires installing Numba and only works in very specific use cases.
  - PredictorX1 3y ago
    Language or platform standardization for data science is over-rated. Most often, analysis is performed remotely, and resulting models and other work product are deployed, "frozen" on production platforms. There is seldom need to match the data scientist's tools to match those of the organization.
- kritr 3y ago
  As someone that doesn’t spend a lot of time in Python, but is familiar with the ecosystem and numpy. I would say this is actually a lot more approachable than I thought it would be. The only thing that really surprised me was that rewriting corrcoef led to a speedup. I would imagine that’s a fairly optimized function in numpy.
- yodelshady 3y ago
  Cool. Now, look at the "rust code" when you're tasked with changing mean to median... in the average non-programming specialist shop, where what you have access to is the production server. Which may or may not be the dev server, or a laptop. No they're not complying to your CI/CD best standards - tell me with a straight face they're learnable in less than a month with a budget of $0 above your existing capital. Yes they're adding real value to the economy. That's "computing for everyone", and one of those tools works for that scenario.
  - freilanzer 3y ago
    If this "specialist shop" is changing Python code directly on the production server then it's not going to be around for long, and deservedly so.
a_bonobo 3y ago
Would be very interested in seeing how torch.compile() fares with OP's numpy portions: https://pytorch.org/blog/compiling-numpy-code/ https://pytorch.org/blog/compiling-numpy-code/
herdcall 3y ago
This seems like a meaningless claim because the core parts of both itertools and pandas libraries used here are written in C, unless I'm missing some context here.
- ivancho 3y ago
  cpython is also written in C
IshKebab 3y ago
Interesting for sure, but that's definitely a more painful optimisation journey than the Rust one, which only involved Rust. No custom JITing of a subset of Rust, or calling out to C. I think the real takeaway from my experience of Rust vs Python performance is that in practice people don't optimise 99% of Python so the real number of significance is the 8x speedup he got from just switching to Rust with no profiling whatsoever. That's actually quite a low difference compared to my experience - I'd say 50x is more typical.
- pjmlp 3y ago
  It would be a different matter if something like PyPy would be the reference implementation, but alas we are quite far from it. As it stands, any AOT/JIT compiled language has an edge over Python, considering using only the reference implementation, which is quite relevant on systems were one cannot expect to allowed to install additional development tools. As for the bindings to native libraries, yeah Python might be favoured for bindings, however those native libraries can be called by any language that has FFI capabilities anyway, alongside their JIT/AOT capabilities.

ndr 3y ago

For context, this is in response to 'Analyzing Data 180,000x Faster with Rust' [0], previously discussed on HN [1] [0] https://willcrichton.net/notes/k-corrset/ https://willcrichton.net/notes/k-corrset/ [1] https://news.ycombinator.com/item?id=37964161 https://news.ycombinator.com/item?id=37964161

orf 3y ago

I’m all for not prematurely optimising and the point of the authors post is still valid, but it is hard to argue that the resulting code is more readable or understandable than the rust code. The rust code is pretty simple, and plain, and doesn’t need any of these tricks

sgarland 3y ago
I don’t think Numpy is complex for anyone doing data analysis. Numba is effectively transparent, since it’s just a decorator. It’s very readable to me (not a data scientist, but I’m decent at Python), YMMV.
datadrivenangel 3y ago
There are benefits to standardizing on a single language and codebase, so it may be worth having some code that is less readable or performant in order to optimize for the global maximum and not prematurely introduce too many languages.
- appplication 3y ago
  My least favorite thing about performance optimizations is how hard they are to guard against regressions with standard CI/CD. Unit tests, integration tests, etc. are rarely the right tools. And in some systems (e.g. distributed data processing), local performance often does not indicate true performance at scale.
  - sgarland 3y ago
    If you want truly accurate results, you have to have a dedicated machine – not a VM (unless you’re controlling the hypervisor, and can guarantee no overcommit), not a container, but an entire machine – and tightly control every aspect of it. Updated packages? All results are invalid until you can prove the baseline hasn’t changed. I don’t think most people need this level of accuracy, but it is something to consider. Database queries are even worse, because not only do you need representative data at a similar scale (or you’ve scaled both hardware and working set accurately), but you have to hope that the table/column stats in prod and test are kept at a similar level of accuracy, which is more art than science. Did the query regress because of the change, or because writes to an unrelated table have increased load and the analysis daemon is now struggling? Did it regress in test because someone else pushed a change to a rarely-used query, causing buffer churn when CI ran your job? Etc.
    - appplication 3y ago
      I agree with all of this. Because of this, even at massive scale it’s hard to justify spending the time and effort needed to get this level of testing.
- FridgeSeal 3y ago
  Conversely, freeing data science/numerical analysis tooling from the tire-fire of the Python ecosystem might actually be a net benefit for the community at large. Get a bit more variety into the options, explore some alternative approaches. Who knows, maybe some ideas from other programming languages will yield some useful advantages.
  - sgarland 3y ago
    Python’s ecosystem is great IFF you use 3rd party tooling carefully, and understand how and when to use it. The biggest one is to use venvs liberally, and to let as few things as possible interact with system Python. If there’s a CLI tool written in Python, ideally install it with pipx. Use Poetry for every project. Unfortunately these aren’t inherently obvious until you have a raging tire fire, and even then, you have to know the source of it and that better options exist. But hey, at least it isn’t Node.
    - pjmlp 3y ago
      Node has a JIT by default, while Python is still fighting to get one that is widely accepted. I still have hopes that Julia and Ruby competition makes it change.
      - sgarland 3y ago
        In theory, Node is faster than Python due to this, among other things. In practice, I have seen Node apps blocking or crashing far more often than Python apps, which drags the performance down just a bit. In the meantime, @njit decorator on Python functions, and boom, done.
        pjmlp 3y ago
        I am yet to see such thing, to each its own I guess. @njit doesn't work in regular Python, requires installing Numba and only works in very specific use cases.
- PredictorX1 3y ago
  Language or platform standardization for data science is over-rated. Most often, analysis is performed remotely, and resulting models and other work product are deployed, "frozen" on production platforms. There is seldom need to match the data scientist's tools to match those of the organization.
kritr 3y ago
As someone that doesn’t spend a lot of time in Python, but is familiar with the ecosystem and numpy. I would say this is actually a lot more approachable than I thought it would be. The only thing that really surprised me was that rewriting corrcoef led to a speedup. I would imagine that’s a fairly optimized function in numpy.
yodelshady 3y ago
Cool. Now, look at the "rust code" when you're tasked with changing mean to median... in the average non-programming specialist shop, where what you have access to is the production server. Which may or may not be the dev server, or a laptop. No they're not complying to your CI/CD best standards - tell me with a straight face they're learnable in less than a month with a budget of $0 above your existing capital. Yes they're adding real value to the economy. That's "computing for everyone", and one of those tools works for that scenario.
- freilanzer 3y ago
  If this "specialist shop" is changing Python code directly on the production server then it's not going to be around for long, and deservedly so.

a_bonobo 3y ago

Would be very interested in seeing how torch.compile() fares with OP's numpy portions: https://pytorch.org/blog/compiling-numpy-code/ https://pytorch.org/blog/compiling-numpy-code/

herdcall 3y ago

This seems like a meaningless claim because the core parts of both itertools and pandas libraries used here are written in C, unless I'm missing some context here.

ivancho 3y ago
cpython is also written in C

IshKebab 3y ago

Interesting for sure, but that's definitely a more painful optimisation journey than the Rust one, which only involved Rust. No custom JITing of a subset of Rust, or calling out to C. I think the real takeaway from my experience of Rust vs Python performance is that in practice people don't optimise 99% of Python so the real number of significance is the 8x speedup he got from just switching to Rust with no profiling whatsoever. That's actually quite a low difference compared to my experience - I'd say 50x is more typical.

pjmlp 3y ago
It would be a different matter if something like PyPy would be the reference implementation, but alas we are quite far from it. As it stands, any AOT/JIT compiled language has an edge over Python, considering using only the reference implementation, which is quite relevant on systems were one cannot expect to allowed to install additional development tools. As for the bindings to native libraries, yeah Python might be favoured for bindings, however those native libraries can be called by any language that has FFI capabilities anyway, alongside their JIT/AOT capabilities.