3 ms·

I don't disagree with any of this but > "Why in the hell would you fork then load, instead of load then fork?" In python it often seems to make little differe

by ris 7y ago

I don't disagree with any of this but > "Why in the hell would you fork then load, instead of load then fork?" In python it often seems to make little difference. The continual refcount incrementing and decrementing sooner or later touches most everything and causes the copy to happen whether you're mutating an object or not. I've had some broad thoughts about how one would give cpython the ability to "turn off" gc and refcounting for some "forever" objects which you know you're never going to want to free, but it wouldn't be pretty as it would require segregating these objects into their own arenas to prevent neighbour writes dirtying the whole page anyway...

wrmsr 7y ago
They took a step towards this with https://docs.python.org/3/library/gc.html#gc.freeze https://docs.python.org/3/library/gc.html#gc.freeze but it doesn't go as far as disabling refcount touching outright. I've experimented with doing that, both per-object and just globally, and the results really were promising if your forkserver can keep up with providing the necessarily much shorter-lived worker processes.
- ris 7y ago
  Thanks for this link - I had completely missed it (I think I was just expecting to disable gc entirely or perform some rudimentary surgery on its linked list)
jks 7y ago
There's a 2017 writeup from Instagram: https://instagram-engineering.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172 https://instagram-engineering.com/dismissing-python-garbage-...
- ris 7y ago
  This isn't quite the same thing, but it is one of the articles that spurred my thoughts on this subject. In cpython, gc != refcounting. Instagram were talking about disabling gc, which would have stopped objects which they weren't using from being falsely copied, but wouldn't have stopped objects that they were using (but not mutating) from being copied.

by ris 7y ago

wrmsr 7y ago

They took a step towards this with https://docs.python.org/3/library/gc.html#gc.freeze https://docs.python.org/3/library/gc.html#gc.freeze but it doesn't go as far as disabling refcount touching outright. I've experimented with doing that, both per-object and just globally, and the results really were promising if your forkserver can keep up with providing the necessarily much shorter-lived worker processes.

ris 7y ago
Thanks for this link - I had completely missed it (I think I was just expecting to disable gc entirely or perform some rudimentary surgery on its linked list)

jks 7y ago

There's a 2017 writeup from Instagram: https://instagram-engineering.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172 https://instagram-engineering.com/dismissing-python-garbage-...

ris 7y ago
This isn't quite the same thing, but it is one of the articles that spurred my thoughts on this subject. In cpython, gc != refcounting. Instagram were talking about disabling gc, which would have stopped objects which they weren't using from being falsely copied, but wouldn't have stopped objects that they were using (but not mutating) from being copied.