3 ms·

I'm curious, anyone have an anecdote where using Python for data processing or machine learning became a performance problem? My experience is that Python has

I'm curious, anyone have an anecdote where using Python for data processing or machine learning became a performance problem? My experience is that Python has so many high quality packages to deal with cpu intensive tasks that unless you absolutely cannot use them, for some reason, performance is rarely a problem. But I'd like to be informed about the exceptions.

c0balt 3y ago
At least for data processing pure python can quickly become a problem when handling somewhat large, dozens of GB, files. I've encountered this a few times and tge way to go is usually to either, if viable, port to a different language or only port the hot parts to a relevant library or Cython/Rust with pyo3.
- layoric 3y ago
  Out of interest, did you find this was a problem that using optimised numpy still didn’t perform good enough? (depending on what you call pure python, yes I agree python loops etc are extremely slow)
  - c0balt 3y ago
    I didn't find any problems with numpy based processing in practice. The issue more often is that users choose Python for ease of writing and the std lib and don't consider numpy for these cases. The performance considerarions usually only come up later.
efxhoy 3y ago
I wrote simulation software at a research department previously. 99.8% of the problems were perfect for numpy/pandas and learning a compiled language wasn’t worth it for the 0.2%. If I was to work the same problems again I’d do it in two years when polars has all the good stuff from pandas and I’d learn enough rust to drop down to it for the custom stuff.
jebarker 3y ago
I've hit problems a couple of times with compute intensive data loading tasks in Pytorch. One involved custom data augmentations on spatial graphs. I couldn't find any python libraries that offered performant implementations of the functions needed and ended up writing custom C++ extensions to call from python. The other involved running video encoders and decoders inside the dataloader. The problem was more Pytorch specific than python per se, but the way Pytorch handles multiprocessing and CUDA streams made efficient parallelism very difficult. It would have been much easier in C++. I've also needed to ditch python for ML inference in production when extremely low latency was required.
nemothekid 3y ago
I rarely see performance problems that are show stoppers, especially in the cloud, where you can always just buy a bigger node. Usually if something is going to be rewritten C++/Rust/$performant_lang it's that high velocity iteration on the codebase has mostly stopped and we could save 50-60% in infrastructure costs by running it on nodes with less resources. Then naturally you start to factor in the engineering manhours for the rewrite vs the potential savings.
electroly 3y ago
Not a single one of our data scientists knows how to use multiprocessing. Their jobs are all 1 CPU core, hundreds of gigabytes of RAM. The EC2 instances they run on end up wasting tons of cores, even with high-memory instances. This isn't necessarily Python's fault, but rather because Python has a lot of users who are not computer programmers. One thing we can blame Python for is that the data scientists can't ever loop over their data with a regular loop. Simply impossible; it's too slow and would immediately wreck performance. They're more Pandas programmers than they are Python programmers; everything has to be done with Pandas operations for performance to be acceptable. This limits our (the computer programmers) ability to help out, because the languages we use are not slow like that, so we may give suggestions from our world that simply can't be used in Python because it's too slow. "Can't you just loop over the data and do XYZ?" Nope, they can't. We still use Python because it's all the data scientists know, and it's easier/cheaper to just write a bigger check to AWS than to hire people with both data science and computer programming backgrounds.
ssivark 3y ago
I've run into some pretty straight-forward problems (eg: modeling physical systems, with linear algebra and some other basic math) where avoiding python for loops meant allocating memory up front for massive numpy, and then doing multiple separate vectorized operations on them rather than fusing all of them into a single for loop. Required several hundred GB of RAM and took a few hours IIRC. This was of course still better than Python for loops... /facepalm I wrote the same thing in Julia in about the same number of lines of code -- avoiding the sins above -- and I could run it on my laptop in a few seconds.
- geysersam 3y ago
  For stuff like that I think Numba really shines. But yes I get the issue. Julia is great, but sometimes a bit less user friendly than Python, it's quite easy to shoot oneself in the foot performance wise using Julia.
  - ssivark 3y ago
    I hate the fact that if I choose Numba, then I can’t easily compose with other tools like autodiff, etc. Maybe Jax improves the situation in some aspects. Generally though… While Python might have many distinct popular/well-developed tools, they tend to compose very badly. IME, it feels relatively painful if you’re trying to do something new which hasn’t been solved and wrapped for you. As for Julia footguns, it has been relatively trivial IME to get substantial improvements compared to Python. Whether that is the best one can do, I dunno.

c0balt 3y ago

At least for data processing pure python can quickly become a problem when handling somewhat large, dozens of GB, files. I've encountered this a few times and tge way to go is usually to either, if viable, port to a different language or only port the hot parts to a relevant library or Cython/Rust with pyo3.

layoric 3y ago
Out of interest, did you find this was a problem that using optimised numpy still didn’t perform good enough? (depending on what you call pure python, yes I agree python loops etc are extremely slow)
- c0balt 3y ago
  I didn't find any problems with numpy based processing in practice. The issue more often is that users choose Python for ease of writing and the std lib and don't consider numpy for these cases. The performance considerarions usually only come up later.

efxhoy 3y ago

I wrote simulation software at a research department previously. 99.8% of the problems were perfect for numpy/pandas and learning a compiled language wasn’t worth it for the 0.2%. If I was to work the same problems again I’d do it in two years when polars has all the good stuff from pandas and I’d learn enough rust to drop down to it for the custom stuff.

jebarker 3y ago

I've hit problems a couple of times with compute intensive data loading tasks in Pytorch. One involved custom data augmentations on spatial graphs. I couldn't find any python libraries that offered performant implementations of the functions needed and ended up writing custom C++ extensions to call from python. The other involved running video encoders and decoders inside the dataloader. The problem was more Pytorch specific than python per se, but the way Pytorch handles multiprocessing and CUDA streams made efficient parallelism very difficult. It would have been much easier in C++. I've also needed to ditch python for ML inference in production when extremely low latency was required.

nemothekid 3y ago

I rarely see performance problems that are show stoppers, especially in the cloud, where you can always just buy a bigger node. Usually if something is going to be rewritten C++/Rust/$performant_lang it's that high velocity iteration on the codebase has mostly stopped and we could save 50-60% in infrastructure costs by running it on nodes with less resources. Then naturally you start to factor in the engineering manhours for the rewrite vs the potential savings.

electroly 3y ago

Not a single one of our data scientists knows how to use multiprocessing. Their jobs are all 1 CPU core, hundreds of gigabytes of RAM. The EC2 instances they run on end up wasting tons of cores, even with high-memory instances. This isn't necessarily Python's fault, but rather because Python has a lot of users who are not computer programmers. One thing we can blame Python for is that the data scientists can't ever loop over their data with a regular loop. Simply impossible; it's too slow and would immediately wreck performance. They're more Pandas programmers than they are Python programmers; everything has to be done with Pandas operations for performance to be acceptable. This limits our (the computer programmers) ability to help out, because the languages we use are not slow like that, so we may give suggestions from our world that simply can't be used in Python because it's too slow. "Can't you just loop over the data and do XYZ?" Nope, they can't. We still use Python because it's all the data scientists know, and it's easier/cheaper to just write a bigger check to AWS than to hire people with both data science and computer programming backgrounds.

ssivark 3y ago

I've run into some pretty straight-forward problems (eg: modeling physical systems, with linear algebra and some other basic math) where avoiding python for loops meant allocating memory up front for massive numpy, and then doing multiple separate vectorized operations on them rather than fusing all of them into a single for loop. Required several hundred GB of RAM and took a few hours IIRC. This was of course still better than Python for loops... /facepalm I wrote the same thing in Julia in about the same number of lines of code -- avoiding the sins above -- and I could run it on my laptop in a few seconds.

geysersam 3y ago
For stuff like that I think Numba really shines. But yes I get the issue. Julia is great, but sometimes a bit less user friendly than Python, it's quite easy to shoot oneself in the foot performance wise using Julia.
- ssivark 3y ago
  I hate the fact that if I choose Numba, then I can’t easily compose with other tools like autodiff, etc. Maybe Jax improves the situation in some aspects. Generally though… While Python might have many distinct popular/well-developed tools, they tend to compose very badly. IME, it feels relatively painful if you’re trying to do something new which hasn’t been solved and wrapped for you. As for Julia footguns, it has been relatively trivial IME to get substantial improvements compared to Python. Whether that is the best one can do, I dunno.