4 ms·

TFA makes it sound like no two Python programs can execute simultaneously on two different cores. I would be very, very surprised if that would hold true for PH

by TekMol 5y ago

TFA makes it sound like no two Python programs can execute simultaneously on two different cores. I would be very, very surprised if that would hold true for PHP or JS. I have the feeling what the TFA really means is something different. But I cannot really figure out what.

Ndymium 5y ago
What the article means is that a single Python OS process can have multiple OS threads but only one of them can execute Python bytecode at any given time. So if you have a multithreaded Python application (on CPython), only one of those threads will run your code at a time while the others are waiting. Naturally you can have multiple OS processes of Python that run in parallel. This is how Python web services usually work. Then, of course, you need to share memory explicitly via some mechanism like Redis or SysV shared memory, so it leads to different design considerations.
ynik 5y ago
You can only have a single core running Python code per process. You can also have a single Python process running multiple threads in parallel, as long as all but one of those threads are currently running C code (numpy, system calls, ...) But if your program spends more than a tiny fraction of time in interpreted Python code, the GIL will slow you done if you try to use all cores. You can have multiple processes running Python at the same time -- and indeed, many Python programs work around the GIL by forking sub-processes. This tends to introduce significant extra complexity into the program (and extra memory usage, as the easiest solution is typically to copy data to all processes).
- Symmetry 5y ago
  If your inputs are immutable and you don't want to copy them that works too since if you're careful. Copy on write means that it won't be duplicated by default. Of course reference counting changes means you will be writing to the inputs by default but if you're just branching for a computation you can temporarily turn off garbage collection while you diverge. Disabling garbage collection for a short lived forked process can even make sense on its own terms.
masklinn 5y ago
> TFA makes it sound like no two Python programs can execute simultaneously on two different cores It does not. It states that two python threads can’t execute bytecode simultanously, the implication is that that’s within a single process (which is the case) otherwise talking about threads makes no sense. > I have the feeling what the TFA really means is something different. But I cannot really figure out what. TFA means exactly what it says. Two threads can’t execute python bytecode in parallel, because there’s a big lock around the main interpreter loop (that is in essence what the GIL is). That is quite obviously within the confines of a single Python interpreter = runtime = process.

by TekMol 5y ago

Ndymium 5y ago

What the article means is that a single Python OS process can have multiple OS threads but only one of them can execute Python bytecode at any given time. So if you have a multithreaded Python application (on CPython), only one of those threads will run your code at a time while the others are waiting. Naturally you can have multiple OS processes of Python that run in parallel. This is how Python web services usually work. Then, of course, you need to share memory explicitly via some mechanism like Redis or SysV shared memory, so it leads to different design considerations.

ynik 5y ago

You can only have a single core running Python code per process. You can also have a single Python process running multiple threads in parallel, as long as all but one of those threads are currently running C code (numpy, system calls, ...) But if your program spends more than a tiny fraction of time in interpreted Python code, the GIL will slow you done if you try to use all cores. You can have multiple processes running Python at the same time -- and indeed, many Python programs work around the GIL by forking sub-processes. This tends to introduce significant extra complexity into the program (and extra memory usage, as the easiest solution is typically to copy data to all processes).

Symmetry 5y ago
If your inputs are immutable and you don't want to copy them that works too since if you're careful. Copy on write means that it won't be duplicated by default. Of course reference counting changes means you will be writing to the inputs by default but if you're just branching for a computation you can temporarily turn off garbage collection while you diverge. Disabling garbage collection for a short lived forked process can even make sense on its own terms.

masklinn 5y ago

> TFA makes it sound like no two Python programs can execute simultaneously on two different cores It does not. It states that two python threads can’t execute bytecode simultanously, the implication is that that’s within a single process (which is the case) otherwise talking about threads makes no sense. > I have the feeling what the TFA really means is something different. But I cannot really figure out what. TFA means exactly what it says. Two threads can’t execute python bytecode in parallel, because there’s a big lock around the main interpreter loop (that is in essence what the GIL is). That is quite obviously within the confines of a single Python interpreter = runtime = process.