2 ms·
This has had me thinking about the Python/malloc story for a little bit. It does make sense CPython is tuned more towards runtime than startup time. One is a c
by shakna 7y ago
This has had me thinking about the Python/malloc story for a little bit.
It does make sense CPython is tuned more towards runtime than startup time. One is a continual cost, one is paid once and doesn't matter for a large variety of workloads.
But if CPython really is make 1000+ allocations during startup, then it is probably going to fragment badly, which slow down everything, and into the future as well...
Seems that's not a crazy idea. [0] Zapier linked CPython to jemalloc which tends to fragment less and saw 40% memory gains.
I guess if you took the time to tune jemalloc to your workload you should be able to see significant performance improvements at startup as well as in the hot path.
[0] https://zapier.com/engineering/celery-python-jemalloc/ https://zapier.com/engineering/celery-python-jemalloc/