2 ms·

> No, no the same thing. Well to me it is the same thing. Two threads sharing state through a common heap, or two processes sharing state through a shared mem

> No, no the same thing. Well to me it is the same thing. Two threads sharing state through a common heap, or two processes sharing state through a shared memory is pretty much indistinguishable, at least on Linux. The question is not multithreading or multiprocessing anymore, the difference to me is more semantic than real. The question is then just how these threads/processes communicate. I would argue that shared mutable state is rarely a good idea, and an equivalent message based system is often preferable. For the few use cases that remain where you would want shared mutable states, as mentioned in my original answer, Python has shared memory support, though with non-built-in types. Improving these shared types should be the only thing you advocate for.

Too 3y ago
Right now, you often get stuck on using multiprocessing because you need to share one object that is unpickleable and can't be sent over queues. Improving shared types would be a great addition. Problem is the reason for objects being unpicklable in the first place, is often a more underlying thing that can't be shared across processes, like open file handles.
miraculixx 3y ago
I agree, actually. It's just that (some) people really want multithreading. And with that Python had the rare chance to say, ok you'll get it, but we will make it safe by default. Now they said ok you get it, even though it's a big risk, we hope it'll work. If not, we'll take it back. In what world is "taking back" even an option?
thayne 3y ago
> two processes sharing state through a shared memory The point is that isn't very practical, especially in python. In practice, any data passed between processes must be serialized and deserialized over a pipe or in shared memory. With a thread per interpreter, native python objecta could potentially be shared across threads/interpreters as long as there was adequate synchronization. Technically, you could also do that with shared memory with multiprocessing, but it would be harder, and you would need to specify the object is intended to be shared from the beginning. > pretty much indistinguishable, at least on Linux. Linux isn't the only OS people use python on. > I would argue that shared mutable state is rarely a good idea, and an equivalent message based system is often preferable. I agree, but even for message passing, having to serialize and deserialize the object across process boundaries hurts performance.

Too 3y ago

Right now, you often get stuck on using multiprocessing because you need to share one object that is unpickleable and can't be sent over queues. Improving shared types would be a great addition. Problem is the reason for objects being unpicklable in the first place, is often a more underlying thing that can't be shared across processes, like open file handles.

miraculixx 3y ago

I agree, actually. It's just that (some) people really want multithreading. And with that Python had the rare chance to say, ok you'll get it, but we will make it safe by default. Now they said ok you get it, even though it's a big risk, we hope it'll work. If not, we'll take it back. In what world is "taking back" even an option?

thayne 3y ago

> two processes sharing state through a shared memory The point is that isn't very practical, especially in python. In practice, any data passed between processes must be serialized and deserialized over a pipe or in shared memory. With a thread per interpreter, native python objecta could potentially be shared across threads/interpreters as long as there was adequate synchronization. Technically, you could also do that with shared memory with multiprocessing, but it would be harder, and you would need to specify the object is intended to be shared from the beginning. > pretty much indistinguishable, at least on Linux. Linux isn't the only OS people use python on. > I would argue that shared mutable state is rarely a good idea, and an equivalent message based system is often preferable. I agree, but even for message passing, having to serialize and deserialize the object across process boundaries hurts performance.