3 ms·
A memory consumption thing that you have to be careful of when doing that that we ran into recently. When you call fork Linux doesn't actually have to copy any
by Symmetry 10y ago
A memory consumption thing that you have to be careful of when doing that that we ran into recently. When you call fork Linux doesn't actually have to copy any of the existing memory until it's changed thanks to the magic of copy-on-write. But often Python will quickly increment or decrement the reference counters on all those bits of memory forcing copies; increasing memory usage and slowing things down. So either being careful with that or turning off garbage collection for the process (if it's short lived) are things to consider there.
- jsmeaton 10y agoThis was discussed recently in an instagram blog post. Did your team also independently reach these conclusions?
- Symmetry 10y agoI'm not sure what the order of events was but I should certainly link that blog post. https://engineering.instagram.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172#.xkfs7jfj6 https://engineering.instagram.com/dismissing-python-garbage-...