3 ms·

Notably, subinterpreters might be able to share data without copying it. ProcessPoolExecutor can only share data by pickling it.

• pdonis 7y ago
  > subinterpreters might be able to share data without copying it In the current proposal as described in the PEP, they wouldn't. Subinterpreters will only be able to communicate through channels, and arbitrary objects can't even be sent through channels; they would have to be pickled or marshaled, similar to sharing data between processes.
  • duckerude 7y ago
    If I'm reading the PEP correctly, it's possible to send memoryview buffers through channels without copying the underlying data. The few other types supported by channels (None, bytes, str, int) are all immutable, so I would guess that those are also shared without copying.
    • pdonis 7y ago
      Yes, these particular types of data can be shared (as you say, sharing immutable objects obviously doesn't require copying since they're immutable); but I took "shared data" to mean being able to share arbitrary objects, since that's the sense of "sharing" that is relevant when we're talking about various forms of concurrency. If all I can share between concurrent computations is those limited data types, then I have to pickle/marshal everything else, which is effectively the same thing I have to do with multiple processes.
      • duckerude 7y ago
        I see. I was paraphrasing from the article originally, and didn't mean "share data" in a very general way. Sharing memoryviews lets you share e.g. numpy arrays, so it should be powerful enough for some applications.