4 ms·

> not a good fit for apps requiring high performance / high scalability anyway It's way too late to artificially limit Python's reach like this. It's already h

> not a good fit for apps requiring high performance / high scalability anyway It's way too late to artificially limit Python's reach like this. It's already heavily used in data science and machine learning. People need performance right now for things that they are already doing in Python. We can't go back to a time when that isn't true. Now it's one of the most common use cases for Python. People are building careers on this.

KeplerBoy 3y ago
Those people/use-cases don't care about the GIL. When you're doing anything numerical you're calling into Numpy, Pytorch or Polars anyways and those libraries make use of multiple threads and drop the GIL. So yes, you can do Matmuls to your heart's content in one python thread and wait for file i/o in another thread without ever running into issues with the GIL. I'd even say if your Python Interpreter isn't bored waiting for stuff to happen, you're doing it wrong.
- shafte 3y ago
  > Those people/use-cases don't care about the GIL. This is not true. The primary funding and motivation for the GIL removal work comes from the numerical computing community. The PEP (https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/) contains direct quotes from folks working on numpy, scipy, PyTorch, scikit-learn, etc. and also practitioners from places like Meta, DeepMind and so on, describing the practical constraints that the GIL places on many workloads.
- deleted 3y ago
  [deleted]
jerf 3y ago
This doesn't solve the data science and machine learning problems, though. You will still have the problem that optimized code paths will be hundreds to thousands to even millions of times faster (on a GPU) than any Python could be, and falling back to pure Python, accidentally or otherwise, will still incur those slowdowns, even if the resulting pure Python code is embarrassingly parallel and that is fully perfectly exploited. Personally, I wouldn't be surprised all this work gets done on GIL-removal, and the end result is that if you run 10 threads in Python you max out at roughly a 3x performance improvement in your pure Python code (and often get less, I do mean that as a max not an average), with all the memory traffic it is doing. Honestly even if it did perfectly parallelize up to 32 cores, a completely impossible absurdity, it would still be noticeably inferior performance to what many languages will give you on one core. I can't help but think that if you're sitting here waiting with bated breath for the GIL-ectomy to improve your Python performance that that is a sign you should be rewriting your code right now in any number of languages that are simply faster. I very, very strongly suspect that this is going to result in very very disappointing speedups when it is done.
- ledauphin 3y ago
  The most obvious performance win for free threading in Python is where you have large read-only, immortal data that is referenced by all threads. The "memory traffic" you mention is a major issue for multiprocessing approaches, and sub-interpreters don't provide access to shared memory in this way, either. The no-gil project has already been benchmarked to scale basically linearly in this scenario, so your hand-waving predictions are already known to be inaccurate. https://www.backblaze.com/blog/python-gil-vs-nogil-boost-i-o-performance-10x-with-one-line-change/ https://www.backblaze.com/blog/python-gil-vs-nogil-boost-i-o...
  - jerf 3y ago
    I'm not talking about the performance that no-GIL is going to get on little bits and pieces of code. I'm talking about the real-world performance it is likely to get on real code. I work in languages that don't have anywhere near the overhead and ceremony that Python does, and when you're writing multithreaded code for the explicit purpose of optimizing some particular bit of slow code, it is quite common to find you completely top out at 3x or 4x performance no matter how many cores you throw at it, because you bottleneck on something other than CPU anyhow. Microbenchmarks in those languages can also get perfectly parallel speedups but that's not what you get in real code. Again, if you're sitting on slow Python code and hoping for the GIL-ectomy to save you, you should just go write it in a different language right now. A perfectly parallel Python will still be slower on a 32 core machine than other languages can be on a single core... and by some margin! The competition I'm benchmarking Python's performance work isn't against other Python approaches, it's against other languages entirely. Subinterpreters are just a concession to the enormous pile of other preconditions Python (and dynamic scripting languages) brings with it and have no bearing on how other languages work; they're a hack around that problem. They're not a solution any other language would consider without those requirements weighing them down.
Too 3y ago
The only hope is for Mojo to deliver on making that Python compatible language compiling straight to the GPU accelerator.
lucasyvas 3y ago
Dynamic/scripting languages will always be slower and will most likely never be suitable for this use case - native libraries will always win and there will be no point to writing these in pure Python. I concur with the parent that I don't believe there's a strong justification here. Personally, I would focus on making Python a top-quality single-threaded experience as well as performance.

KeplerBoy 3y ago

Those people/use-cases don't care about the GIL. When you're doing anything numerical you're calling into Numpy, Pytorch or Polars anyways and those libraries make use of multiple threads and drop the GIL. So yes, you can do Matmuls to your heart's content in one python thread and wait for file i/o in another thread without ever running into issues with the GIL. I'd even say if your Python Interpreter isn't bored waiting for stuff to happen, you're doing it wrong.

shafte 3y ago
> Those people/use-cases don't care about the GIL. This is not true. The primary funding and motivation for the GIL removal work comes from the numerical computing community. The PEP (https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/) contains direct quotes from folks working on numpy, scipy, PyTorch, scikit-learn, etc. and also practitioners from places like Meta, DeepMind and so on, describing the practical constraints that the GIL places on many workloads.
deleted 3y ago
[deleted]

jerf 3y ago

This doesn't solve the data science and machine learning problems, though. You will still have the problem that optimized code paths will be hundreds to thousands to even millions of times faster (on a GPU) than any Python could be, and falling back to pure Python, accidentally or otherwise, will still incur those slowdowns, even if the resulting pure Python code is embarrassingly parallel and that is fully perfectly exploited. Personally, I wouldn't be surprised all this work gets done on GIL-removal, and the end result is that if you run 10 threads in Python you max out at roughly a 3x performance improvement in your pure Python code (and often get less, I do mean that as a max not an average), with all the memory traffic it is doing. Honestly even if it did perfectly parallelize up to 32 cores, a completely impossible absurdity, it would still be noticeably inferior performance to what many languages will give you on one core. I can't help but think that if you're sitting here waiting with bated breath for the GIL-ectomy to improve your Python performance that that is a sign you should be rewriting your code right now in any number of languages that are simply faster. I very, very strongly suspect that this is going to result in very very disappointing speedups when it is done.

ledauphin 3y ago
The most obvious performance win for free threading in Python is where you have large read-only, immortal data that is referenced by all threads. The "memory traffic" you mention is a major issue for multiprocessing approaches, and sub-interpreters don't provide access to shared memory in this way, either. The no-gil project has already been benchmarked to scale basically linearly in this scenario, so your hand-waving predictions are already known to be inaccurate. https://www.backblaze.com/blog/python-gil-vs-nogil-boost-i-o-performance-10x-with-one-line-change/ https://www.backblaze.com/blog/python-gil-vs-nogil-boost-i-o...
- jerf 3y ago
  I'm not talking about the performance that no-GIL is going to get on little bits and pieces of code. I'm talking about the real-world performance it is likely to get on real code. I work in languages that don't have anywhere near the overhead and ceremony that Python does, and when you're writing multithreaded code for the explicit purpose of optimizing some particular bit of slow code, it is quite common to find you completely top out at 3x or 4x performance no matter how many cores you throw at it, because you bottleneck on something other than CPU anyhow. Microbenchmarks in those languages can also get perfectly parallel speedups but that's not what you get in real code. Again, if you're sitting on slow Python code and hoping for the GIL-ectomy to save you, you should just go write it in a different language right now. A perfectly parallel Python will still be slower on a 32 core machine than other languages can be on a single core... and by some margin! The competition I'm benchmarking Python's performance work isn't against other Python approaches, it's against other languages entirely. Subinterpreters are just a concession to the enormous pile of other preconditions Python (and dynamic scripting languages) brings with it and have no bearing on how other languages work; they're a hack around that problem. They're not a solution any other language would consider without those requirements weighing them down.

Too 3y ago

The only hope is for Mojo to deliver on making that Python compatible language compiling straight to the GPU accelerator.

lucasyvas 3y ago

Dynamic/scripting languages will always be slower and will most likely never be suitable for this use case - native libraries will always win and there will be no point to writing these in pure Python. I concur with the parent that I don't believe there's a strong justification here. Personally, I would focus on making Python a top-quality single-threaded experience as well as performance.