3 ms·

For a while I had it in my mind that maybe you could save some memory while having safety just by using os.fork() after pre-importing a bunch of stuff; but then

by ianb 16y ago

For a while I had it in my mind that maybe you could save some memory while having safety just by using os.fork() after pre-importing a bunch of stuff; but then reference counting was pointed out to me and I realized it would be pointless -- the act of merely looking at an object changes the object, so copy-on-write means copy-everything. Reference counting seems like it could be removed... except for all the C extensions that are specifically handling reference counting internally, removal of which I'm guessing would cause a lot of breakage. But then... it seems like those C extensions are exactly the problem, so if PyPy tries to keep them is it messing itself up? (Though I'm guessing it would somehow use proxy objects or some other trick to keep its internal object representation flexible.)

jjs 16y ago
Have your cake and eat it too: open a socket to a CPython process and let it handle the legacy C extensions.
- jnoller 16y ago
  http://morepypy.blogspot.com/2010/04/using-cpython-extension-modules-with.html http://morepypy.blogspot.com/2010/04/using-cpython-extension...
jnoller 16y ago
Ah! But you see, it's possible to trick CPython extensions into thinking you do support refcounting: http://code.google.com/p/ironclad/ http://code.google.com/p/ironclad/ is proof. So you can be compatible, yet advanced :)
1amzave 16y ago
> ...reference counting was pointed out to me and I realized it would be pointless -- the act of merely looking at an object changes the object, so copy-on-write means copy-everything. Hmm...might it be worthwhile to work around this by storing reference counts in a separate set of pages, and having each object instead hold a constant pointer to its refcount? Presumably the extra level of indirection would have some degree of performance impact, but if it allows CoW to actually work in a meaningful way... Though I guess that would probably break every C extension in existence, no? (Note: I know absolutely nothing about Python's C extension API, just guessing.)

by ianb 16y ago

jjs 16y ago

Have your cake and eat it too: open a socket to a CPython process and let it handle the legacy C extensions.

jnoller 16y ago
http://morepypy.blogspot.com/2010/04/using-cpython-extension-modules-with.html http://morepypy.blogspot.com/2010/04/using-cpython-extension...

jnoller 16y ago

Ah! But you see, it's possible to trick CPython extensions into thinking you do support refcounting: http://code.google.com/p/ironclad/ http://code.google.com/p/ironclad/ is proof. So you can be compatible, yet advanced :)

1amzave 16y ago

> ...reference counting was pointed out to me and I realized it would be pointless -- the act of merely looking at an object changes the object, so copy-on-write means copy-everything. Hmm...might it be worthwhile to work around this by storing reference counts in a separate set of pages, and having each object instead hold a constant pointer to its refcount? Presumably the extra level of indirection would have some degree of performance impact, but if it allows CoW to actually work in a meaningful way... Though I guess that would probably break every C extension in existence, no? (Note: I know absolutely nothing about Python's C extension API, just guessing.)