4 ms·

I think that GIL is pretty tangential to multi-core. If you want something to execute in parallel on SMP system it is always better to have it in separate addre

by dfox 16y ago

I think that GIL is pretty tangential to multi-core. If you want something to execute in parallel on SMP system it is always better to have it in separate address spaces. This is even more true in dynamic language runtime, because almost anything you do (such as accessing global namespace) requires some kind of synchronization and for CPU-intensive code slowdown from this synchronisation tends to be larger than from sequential execution with GIL for almost any significant number of concurrent threads (such as 3 threads on Core 2 Quad on Linux 2.6.2something)

blasdel 16y ago
The people making serious real-world efforts at this kind of thing have problems bigger than one machine -- they need to be distributed anyway, so why not focus on multi-process solutions?
- fauigerzigerk 16y ago
  How do you imagine the design of individual processes participating in that distributed architecture? Each of them is going to have to keep a lot of data in memory and allow concurrent access to it. Your argument is like saying we don't have to bother with wheels on aircraft because flying is a much bigger problem than rolling on wheels.
runT1ME 16y ago
> requires some kind of synchronization and for CPU-intensive code slowdown from this synchronisation tends to be larger than from sequential execution with GIL for almost any significant number of concurrent threads (such as 3 threads on Core 2 Quad on Linux 2.6.2something) what? You're saying the absolute worst case synchronization scenario is on par with the GIL, so you might as well not have it?
- deleted 16y ago
  [deleted]
- dfox 16y ago
  Exactly, because in other cases the GIL does not matter (in best case both approaches would be on par, which they probably would not, due to greater lock contention in finely grained case). Trying to optimize synchronization on so fine level as is done inside various kernels (Linux) would not help, because it is very hard to do correctly in user space ("very hard" as in "probably impossible"). Only way that I see is to try to optimize away majority of shared memory accesses, which would require clever compiler and probably slight modification of Python semantics, or you can simply write your code that is so easily parallelizable as separate processes not threads.

by dfox 16y ago

blasdel 16y ago

The people making serious real-world efforts at this kind of thing have problems bigger than one machine -- they need to be distributed anyway, so why not focus on multi-process solutions?

fauigerzigerk 16y ago
How do you imagine the design of individual processes participating in that distributed architecture? Each of them is going to have to keep a lot of data in memory and allow concurrent access to it. Your argument is like saying we don't have to bother with wheels on aircraft because flying is a much bigger problem than rolling on wheels.

runT1ME 16y ago

> requires some kind of synchronization and for CPU-intensive code slowdown from this synchronisation tends to be larger than from sequential execution with GIL for almost any significant number of concurrent threads (such as 3 threads on Core 2 Quad on Linux 2.6.2something) what? You're saying the absolute worst case synchronization scenario is on par with the GIL, so you might as well not have it?

deleted 16y ago
[deleted]
dfox 16y ago
Exactly, because in other cases the GIL does not matter (in best case both approaches would be on par, which they probably would not, due to greater lock contention in finely grained case). Trying to optimize synchronization on so fine level as is done inside various kernels (Linux) would not help, because it is very hard to do correctly in user space ("very hard" as in "probably impossible"). Only way that I see is to try to optimize away majority of shared memory accesses, which would require clever compiler and probably slight modification of Python semantics, or you can simply write your code that is so easily parallelizable as separate processes not threads.