4 ms·

Yeah, I'm worried too. It seems like the core team doesn't share my sense that nogil is the single most important thing that Python needs to do in a world of m

by __d 4y ago

Yeah, I'm worried too. It seems like the core team doesn't share my sense that nogil is the single most important thing that Python needs to do in a world of many-core processors. If nogil is now blocked because it's deemed an unacceptable performance hit vs 3.11, I'll be very, very disappointed.

O5vYtytb 4y ago
Most python code in the world is currently single threaded. It makes sense to me to start with the low hanging fruit that affects the majority of existing code and then move on to harder problems.
- __d 4y ago
  Sure, but Guido put a pre-condition on GIL removal years ago that it must not cause single-threaded code to suffer a performance penalty. So Sam came up with a way to ensure that (on balance, if not in every example), we could have same-or-better performance and no GIL. Awesome! But now we're getting the performance boosts without the GIL removal, and so in the next release, the GIL removal will cause a performance regression unless we can somehow find more performance boosts. It feels like this could just happen forever.
  - O5vYtytb 4y ago
    There's many more improvements to be made in single threaded performance, including things like JIT. Future improvements are complicated, they won't be a free lunch like this first round has generally been. My feeling is that things like JIT and nogil will start out as optional and work their way into the ecosystem over time where the performance implications are less severe for the general use case.
zackees 4y ago
Why? Python gives you the ability to spawn processes so there's always an escape hatch for building multi-core code.
- __d 4y ago
  Right. But as you implicitly note, it's a work-around ("escape hatch"), not a feature. eg. Unix had processes from day 1, but threads were added (with significant effort) because threads are a better abstraction for addressing lot of problems. Especially so when your CPU has lots of cores. The work to make multiprocessing better is certainly useful, and valuable, but it's still a work-around.

by __d 4y ago

O5vYtytb 4y ago

Most python code in the world is currently single threaded. It makes sense to me to start with the low hanging fruit that affects the majority of existing code and then move on to harder problems.

__d 4y ago
Sure, but Guido put a pre-condition on GIL removal years ago that it must not cause single-threaded code to suffer a performance penalty. So Sam came up with a way to ensure that (on balance, if not in every example), we could have same-or-better performance and no GIL. Awesome! But now we're getting the performance boosts without the GIL removal, and so in the next release, the GIL removal will cause a performance regression unless we can somehow find more performance boosts. It feels like this could just happen forever.
- O5vYtytb 4y ago
  There's many more improvements to be made in single threaded performance, including things like JIT. Future improvements are complicated, they won't be a free lunch like this first round has generally been. My feeling is that things like JIT and nogil will start out as optional and work their way into the ecosystem over time where the performance implications are less severe for the general use case.

zackees 4y ago

Why? Python gives you the ability to spawn processes so there's always an escape hatch for building multi-core code.

__d 4y ago
Right. But as you implicitly note, it's a work-around ("escape hatch"), not a feature. eg. Unix had processes from day 1, but threads were added (with significant effort) because threads are a better abstraction for addressing lot of problems. Especially so when your CPU has lots of cores. The work to make multiprocessing better is certainly useful, and valuable, but it's still a work-around.