3 ms·

Removing GIL only increases complexity.

dagmx 2y ago
For the runtime, but not the language
bunderbunder 2y ago
But they've worked very hard at shielding most users from that complexity. And the end result - making multithreading a truly viable alternative to multiprocessing for typical use cases - will open up many opportunities for Python users to simplify their software designs. I suppose only time will tell if that effort succeeds. But the intent is promising.
- dangom 2y ago
  Do you have any references or examples that describe how this simplification would come about? Would love to learn more about it.
  - bunderbunder 2y ago
    See the motivation section of the PEP for a summary with citations for further reading: https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/
pkkm 2y ago
It definitely does, but don't you think that it could be worth it if it makes multithreading usable for CPU-heavy tasks?
- tightbookkeeper 2y ago
  No. Python is orders of magnitude slower than even C# or Java. It’s doing hash table lookups per variable access. I would write a separate program to do the number crunching. Everyone must now pay the mental cost of multithreading for the chance that you might want to optimize something.
  - int_19h 2y ago
    Python doesn't do hash table lookups for local variable access. This only applies to globals and attributes of Python classes that don't use __slots__. The mental cost of multithreading is there regardless because GIL is usually at the wrong granularity for data consistency. That is, it ensures that e.g. adding or deleting a single element to a dict happens atomically, but more often than not, you have a sequence of operations like that which need to be locked. In practice, in any scenario where your data is shared across threads, the only sane thing is to use explicit locks already.
  - zbentley 2y ago
    > It’s doing hash table lookups per variable access. That hasn't been true for many variable accesses for a very long time. LOAD_FAST, LOAD_CONST, and (sometimes) LOAD_DEREF provide references to variables via pointer offset + chasing, often with caches in front to reduce struct instantiations as well. No hashing is performed. Those access mechanisms account for the vast majority (in my experience; feel free to check by "dis"ing code yourself) of Python code that isn't using locals()/globals()/eval()/exec() tricks. The remaining small minority I've seen is doing weird rebinding/shadowing stuff with e.g. closures and prebound exception captures. https://github.com/python/cpython/blob/10094a533a947b72d01ed8195dcf540f2e7820ea/Python/ceval.c#L2529 https://github.com/python/cpython/blob/10094a533a947b72d01ed... https://github.com/python/cpython/blob/10094a533a947b72d01ed8195dcf540f2e7820ea/Objects/frameobject.c#L1969 https://github.com/python/cpython/blob/10094a533a947b72d01ed... So too for object field accesses; slotted classes significantly improve field lookup cost, though unlike LOAD_FAST users have to explicitly opt into slotting. Don't get me wrong, there are some pretty regrettably ordinary behaviors that Python makes much slower than they need to be (per-binding method refcounting comes to mind, though I hear that's going to be improved). But the old saw of "everything is a dict in python, even variable lookups use hashing!" has been incorrect for years.
    - tightbookkeeper 2y ago
      Thanks for the correction and technical detail. I’m not saying this is bad, it’s just the nature of this kind of dynamic language. Productivity over performance.
EasyMark 2y ago
“Make things as simple as possible, but no simpler”, I for one am glad they’ll be letting us use modern CPUs much more easily instead of it being designed around 1998 cpus

For the runtime, but not the language

But they've worked very hard at shielding most users from that complexity. And the end result - making multithreading a truly viable alternative to multiprocessing for typical use cases - will open up many opportunities for Python users to simplify their software designs. I suppose only time will tell if that effort succeeds. But the intent is promising.

dangom 2y ago
Do you have any references or examples that describe how this simplification would come about? Would love to learn more about it.
- bunderbunder 2y ago
  See the motivation section of the PEP for a summary with citations for further reading: https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/

pkkm 2y ago

It definitely does, but don't you think that it could be worth it if it makes multithreading usable for CPU-heavy tasks?

tightbookkeeper 2y ago
No. Python is orders of magnitude slower than even C# or Java. It’s doing hash table lookups per variable access. I would write a separate program to do the number crunching. Everyone must now pay the mental cost of multithreading for the chance that you might want to optimize something.
- int_19h 2y ago
  Python doesn't do hash table lookups for local variable access. This only applies to globals and attributes of Python classes that don't use __slots__. The mental cost of multithreading is there regardless because GIL is usually at the wrong granularity for data consistency. That is, it ensures that e.g. adding or deleting a single element to a dict happens atomically, but more often than not, you have a sequence of operations like that which need to be locked. In practice, in any scenario where your data is shared across threads, the only sane thing is to use explicit locks already.
- zbentley 2y ago
  > It’s doing hash table lookups per variable access. That hasn't been true for many variable accesses for a very long time. LOAD_FAST, LOAD_CONST, and (sometimes) LOAD_DEREF provide references to variables via pointer offset + chasing, often with caches in front to reduce struct instantiations as well. No hashing is performed. Those access mechanisms account for the vast majority (in my experience; feel free to check by "dis"ing code yourself) of Python code that isn't using locals()/globals()/eval()/exec() tricks. The remaining small minority I've seen is doing weird rebinding/shadowing stuff with e.g. closures and prebound exception captures. https://github.com/python/cpython/blob/10094a533a947b72d01ed8195dcf540f2e7820ea/Python/ceval.c#L2529 https://github.com/python/cpython/blob/10094a533a947b72d01ed... https://github.com/python/cpython/blob/10094a533a947b72d01ed8195dcf540f2e7820ea/Objects/frameobject.c#L1969 https://github.com/python/cpython/blob/10094a533a947b72d01ed... So too for object field accesses; slotted classes significantly improve field lookup cost, though unlike LOAD_FAST users have to explicitly opt into slotting. Don't get me wrong, there are some pretty regrettably ordinary behaviors that Python makes much slower than they need to be (per-binding method refcounting comes to mind, though I hear that's going to be improved). But the old saw of "everything is a dict in python, even variable lookups use hashing!" has been incorrect for years.
  - tightbookkeeper 2y ago
    Thanks for the correction and technical detail. I’m not saying this is bad, it’s just the nature of this kind of dynamic language. Productivity over performance.

EasyMark 2y ago

“Make things as simple as possible, but no simpler”, I for one am glad they’ll be letting us use modern CPUs much more easily instead of it being designed around 1998 cpus