3 ms·

> It's simply not "sane multithreading" not being able to run Python (byte)code concurrentltly Please read my post again. I am advocating a GIL per-thread mode

> It's simply not "sane multithreading" not being able to run Python (byte)code concurrentltly Please read my post again. I am advocating a GIL per-thread model with an explicit feature for sharing selected objects. This allows for all cores concurrency, essentially like free threading yet with safeguards. I like to call that a sane way because it builds on decades of research and industry experience of the software engineering community. > As to the PEP itself: It's optional, with the GIL being enabled by the default. It's not optional if you build tools and libraries that need be able to run with both gil and no-gil.

Phil_Latio 3y ago
> I am advocating a GIL per-thread model with an explicit feature for sharing selected objects Is there a PEP or something written about it? I don't like this idea at first glance. Feels like another hack. Though I have to admit it sounds better than the annoying multiprocessing approach. > It's not optional if you build tools and libraries that need be able to run with both gil and no-gil. Why do they need to? A library dev is free not support one or the other? If a library then does not provide the preferred mode of the user, then that's bad, but that's life. Ideally most libraries will support a gil version (that's still the default anyway!), but provide a no-gil version as a bonus. For example, Pytorch could provide thread-based data loaders if the no-gil version is in use.
- lmz 3y ago
  >> I am advocating a GIL per-thread model with an explicit feature for sharing selected objects > Is there a PEP or something written about it? I don't like this idea at first glance. Feels like another hack. Though I have to admit it sounds better than the annoying multiprocessing approach. This seems very similar to Perl's ithreads model: https://perldoc.perl.org/perlthrtut#Shared-And-Unshared-Data https://perldoc.perl.org/perlthrtut#Shared-And-Unshared-Data
- kzrdude 3y ago
  It's basically becoming reality already, PEP 684 per-interpreter gil is the required structure for this (coming in Python 3.12), then only the interface on top of that remains to be exposed, see WIP like https://github.com/jsbueno/extrainterpreters https://github.com/jsbueno/extrainterpreters The full interface to separately-locked interpreter threads is coming in <some future version of python>, independent of PEP 703 work.

Phil_Latio 3y ago

> I am advocating a GIL per-thread model with an explicit feature for sharing selected objects Is there a PEP or something written about it? I don't like this idea at first glance. Feels like another hack. Though I have to admit it sounds better than the annoying multiprocessing approach. > It's not optional if you build tools and libraries that need be able to run with both gil and no-gil. Why do they need to? A library dev is free not support one or the other? If a library then does not provide the preferred mode of the user, then that's bad, but that's life. Ideally most libraries will support a gil version (that's still the default anyway!), but provide a no-gil version as a bonus. For example, Pytorch could provide thread-based data loaders if the no-gil version is in use.

lmz 3y ago
>> I am advocating a GIL per-thread model with an explicit feature for sharing selected objects > Is there a PEP or something written about it? I don't like this idea at first glance. Feels like another hack. Though I have to admit it sounds better than the annoying multiprocessing approach. This seems very similar to Perl's ithreads model: https://perldoc.perl.org/perlthrtut#Shared-And-Unshared-Data https://perldoc.perl.org/perlthrtut#Shared-And-Unshared-Data
kzrdude 3y ago
It's basically becoming reality already, PEP 684 per-interpreter gil is the required structure for this (coming in Python 3.12), then only the interface on top of that remains to be exposed, see WIP like https://github.com/jsbueno/extrainterpreters https://github.com/jsbueno/extrainterpreters The full interface to separately-locked interpreter threads is coming in <some future version of python>, independent of PEP 703 work.