3 ms·

What happens if you replace the glibc malloc used in implementations of higher level languages like python/ruby/js? Aren’t these languages big on allocating sma

What happens if you replace the glibc malloc used in implementations of higher level languages like python/ruby/js? Aren’t these languages big on allocating small objects very often?

kev009 8y ago
It works fine. jemalloc is the system malloc on FreeBSD.
inopinatus 8y ago
Large rails apps can see significant improvement in memory use efficiency with jemalloc e.g. as in https://www.levups.com/en/blog/2017/optimize_ruby_memory_usage_jemalloc_heroku_scalingo.html https://www.levups.com/en/blog/2017/optimize_ruby_memory_usa... However I’ve also seen an improvement from simply setting MALLOC_ARENA_MAX=2.
_wmd 8y ago
Python's allocator makes use of the fact the GIL must be held to avoid any locking in the normal case for objects under 512 bytes. The introduction of pymalloc was a huge perf win at the time. For larger allocations, it's not so clear if a drop-in malloc would be such a win: many allocators just fall back to mmap() for big allocations and so I imagine their behaviour would be quite similar.
ksec 8y ago
There has been lots of discussion lately about Ruby and malloc. jemalloc gives the best results for Server Performance and memory usage [1], but is slightly slower in low memory usage scenario i.e Non Server usage which is popular in Japan. But jemalloc sometimes ( Not sure if this is still the case ) doesn't work well with muslc, used in Alpine Linux. The Ruby Core team has decided against [2] shipping jemalloc by default, but will use it when it is available. Which ultimately led to some work on SleepyGC [3] and Transient heap [4]. [1] https://www.mikeperham.com/2018/04/25/taming-rails-memory-bloat/ https://www.mikeperham.com/2018/04/25/taming-rails-memory-bl... [2] https://bugs.ruby-lang.org/issues/14718 https://bugs.ruby-lang.org/issues/14718 [3] https://bugs.ruby-lang.org/issues/14723 https://bugs.ruby-lang.org/issues/14723 [4] https://bugs.ruby-lang.org/issues/14858 https://bugs.ruby-lang.org/issues/14858

What happens if you replace the glibc malloc used in implementations of higher level languages like python/ruby/js? Aren’t these languages big on allocating small objects very often?

kev009 8y ago

It works fine. jemalloc is the system malloc on FreeBSD.

inopinatus 8y ago

Large rails apps can see significant improvement in memory use efficiency with jemalloc e.g. as in https://www.levups.com/en/blog/2017/optimize_ruby_memory_usage_jemalloc_heroku_scalingo.html https://www.levups.com/en/blog/2017/optimize_ruby_memory_usa... However I’ve also seen an improvement from simply setting MALLOC_ARENA_MAX=2.

_wmd 8y ago

Python's allocator makes use of the fact the GIL must be held to avoid any locking in the normal case for objects under 512 bytes. The introduction of pymalloc was a huge perf win at the time. For larger allocations, it's not so clear if a drop-in malloc would be such a win: many allocators just fall back to mmap() for big allocations and so I imagine their behaviour would be quite similar.

ksec 8y ago

There has been lots of discussion lately about Ruby and malloc. jemalloc gives the best results for Server Performance and memory usage [1], but is slightly slower in low memory usage scenario i.e Non Server usage which is popular in Japan. But jemalloc sometimes ( Not sure if this is still the case ) doesn't work well with muslc, used in Alpine Linux. The Ruby Core team has decided against [2] shipping jemalloc by default, but will use it when it is available. Which ultimately led to some work on SleepyGC [3] and Transient heap [4]. [1] https://www.mikeperham.com/2018/04/25/taming-rails-memory-bloat/ https://www.mikeperham.com/2018/04/25/taming-rails-memory-bl... [2] https://bugs.ruby-lang.org/issues/14718 https://bugs.ruby-lang.org/issues/14718 [3] https://bugs.ruby-lang.org/issues/14723 https://bugs.ruby-lang.org/issues/14723 [4] https://bugs.ruby-lang.org/issues/14858 https://bugs.ruby-lang.org/issues/14858