3 ms·

Oh I was thinking from a native python object perspective. You could have a rule that the refcount must be 1 when sending an object between subinterpreters. I

Oh I was thinking from a native python object perspective. You could have a rule that the refcount must be 1 when sending an object between subinterpreters. In other words, you cannot use an object that was .send() to another subinterpreter. Then you invalidate the reference in that subinterpreter when it calls send to the other subinterpreter which is transferred by assignment. Can transfer any amount of data with zero copies.

int_19h 4y ago
Even a completely empty object will contain a reference to its type, which is itself an object. How will you marshal that? Bear in mind that each subinterpreter has its own copy of type objects, and there isn't even a guarantee that those types match even if their names do.
- samsquire 4y ago
  It sounds it is a problem but I feel it can be solved with engineering and mathematics. Not saying it would be easy though. Couldn't you separate the storage of the refcounts from the objects and use a map to get at them? As for the identities between types being different. To create an subinterpreter that can marshall between subinterpreters without copying the data structures requires a different data structure that is safe to use from any interpreter. We need to decouple the book keeping data structures from the underlying data. We can regenerate book keeping data structures during a .send or .receive Maintaining identity equivalence is an interesting problem. I think it's a more fundamental problem that probably has mathematical solutions. If we think of objects as being a vector in mathematical space. We have pointers between vectors in this memory space. For a data structure to be position independent. We need some way of intending references to be global. But we don't want to introduce a layer of indirection on the reference of object relationships. That would be slower. Could use an atomic counter to ensure that identifiers are globally unique. Don't want to serialize the access to global types. It sounds to me it is a many-to-many to many-to-many problem. Which even databases don't solve very well.
- samsquire 4y ago
  It occurred to me, that I was told about Unison Lang recently and this language uses content addressable functions. In other words, the code for a function is hashed and that is its identity that never changes while the program is running. If we use the same approach with Python, each object could have a hash that corresponds to the code only, instead of the data. This is the objects identity even when added to the book keeping data of another subinterpreter. This requires decoupling of book keeping information from actual object storage. But replaces pointers with lookups which could be inlined to pointer arithmetic.

int_19h 4y ago

Even a completely empty object will contain a reference to its type, which is itself an object. How will you marshal that? Bear in mind that each subinterpreter has its own copy of type objects, and there isn't even a guarantee that those types match even if their names do.

samsquire 4y ago
It sounds it is a problem but I feel it can be solved with engineering and mathematics. Not saying it would be easy though. Couldn't you separate the storage of the refcounts from the objects and use a map to get at them? As for the identities between types being different. To create an subinterpreter that can marshall between subinterpreters without copying the data structures requires a different data structure that is safe to use from any interpreter. We need to decouple the book keeping data structures from the underlying data. We can regenerate book keeping data structures during a .send or .receive Maintaining identity equivalence is an interesting problem. I think it's a more fundamental problem that probably has mathematical solutions. If we think of objects as being a vector in mathematical space. We have pointers between vectors in this memory space. For a data structure to be position independent. We need some way of intending references to be global. But we don't want to introduce a layer of indirection on the reference of object relationships. That would be slower. Could use an atomic counter to ensure that identifiers are globally unique. Don't want to serialize the access to global types. It sounds to me it is a many-to-many to many-to-many problem. Which even databases don't solve very well.
samsquire 4y ago
It occurred to me, that I was told about Unison Lang recently and this language uses content addressable functions. In other words, the code for a function is hashed and that is its identity that never changes while the program is running. If we use the same approach with Python, each object could have a hash that corresponds to the code only, instead of the data. This is the objects identity even when added to the book keeping data of another subinterpreter. This requires decoupling of book keeping information from actual object storage. But replaces pointers with lookups which could be inlined to pointer arithmetic.