2 ms·
> But a shared memory segment python with a stable binary object format doesn't exist There is multiprocessing.Queue (https://docs.python.org/3/library/multipr
by spacechild1 3y ago
> But a shared memory segment python with a stable binary object format doesn't exist
There is multiprocessing.Queue (https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Queue https://docs.python.org/3/library/multiprocessing.html#multi...).
I don't know if it uses shared memory, or rather sockets or pipes, but this is just an implementation detail.
My point is that there is no fundamental difference between isolated interpreters and processes when it comes to data sharing. Either way, you need a (binary) serialization format and some thread/process-safe queue.
I would have naively assumed that you could repurpose multiprocessing.Queue for passing data between multiple interpreters; you would just need to replace the underlying communication mechanism (sockets, pipes, shared memory, whatever it is) with a queue + mutex. But then again, I'm not familiar with the actual code base. If there are any complications that I didn't take into acccount, I would be curious to hear about them.
Interestingly, the PEP authors currently don't propose an API for exchanging data and instead suggest using raw pipes:
> https://peps.python.org/pep-0554/#api-for-sharing-data https://peps.python.org/pep-0554/#api-for-sharing-data
Of course, this is just a temporary hack. It would be ridiculous to use actual pipes for sharing data within the same process...