2 ms·
There's still plenty of programs where you'd want to spawn a thread that basically doesn't touch any of the data of the parent thread. It'd be nice if there was
by Doxin 6y ago
There's still plenty of programs where you'd want to spawn a thread that basically doesn't touch any of the data of the parent thread. It'd be nice if there was a way around the GIL for those cases.
As an example take the multiprocessing.pool.ThreadPool.map function. Most of the use-cases of that function only read from the parent threads memory once to pass the function arguments. After that the thread may very well spend a lot of time only reading memory it reserved itself. It's rather wasteful to have that thread wait on the GIL.
Of course I haven't got an obvious solution either but it seems to me that for at least a subset of the uses of threading you could work around the GIL without breaking python too badly.
Perhaps split the GIL into a per-thread Thread-interpreter-lock. That way each object can simply be annotated with which lock it belongs to. That way you still get proper atomicity like you're used to in python, but a thread might also actually run concurrently a lot of the time if it only touches objects it created itself.
- loeg 6y ago> There's still plenty of programs where you'd want to spawn a thread that basically doesn't touch any of the data of the parent thread. It'd be nice if there was a way around the GIL for those cases. Subinterpreters gets you that (eventually; not in v1 apparently). > Perhaps split the GIL into a per-thread Thread-interpreter-lock. That way each object can simply be annotated with which lock it belongs to. That way you still get proper atomicity like you're used to in python, but a thread might also actually run concurrently a lot of the time if it only touches objects it created itself. I don't think this scheme would work especially well (or not better than subinterpreters). Message passing still requires copying, or some way to quiesce other threads and recursively change lock ownership of an object graph. To avoid deadlocks, at least one thread will need to drop its own lock to acquire the thread lock of objects owned by a 2nd thread. I'm not sure you could do that and preserve the legacy global atomicity behavior, and it would by definition be slower and bloatier on any individual thread than the existing GIL behavior.