4 ms·
But instead of cross-process serialization we get cross-interpreter serialization, so serialization shenanigans don't go away. There is support in the newest p
by ptx 3y ago
But instead of cross-process serialization we get cross-interpreter serialization, so serialization shenanigans don't go away.
There is support in the newest pickle protocol for using a shared buffer to transfer data more efficiently, but that would work in multiprocessing [0] just as well as in subinterpreters (and currently isn't implemented in either one).
[0] https://github.com/python/cpython/issues/89467 https://github.com/python/cpython/issues/89467
- sitkack 3y agoI agree in the short term, those problems aren't entirely solved, but it does provide for an eventual serialization free way to communicate between threads. Queues, immutable records, atomic refcounts, a global object heap for shared items. There are lots of way forward here that don't involve a full SERDES round trip.