5 ms·

The author never made clear if they are measuring virtual memory usage, or physical memory usage. Having a lot of virtual memory does not "cost" much: it's just

The author never made clear if they are measuring virtual memory usage, or physical memory usage. Having a lot of virtual memory does not "cost" much: it's just having permission to use a lot of memory if you so wish. It's possible to have huge chunks of your virtual address space with no physical memory backing them. Physical memory is the amount of physical RAM your process is using. Memory managers tend to greedily request a lot of virtual memory because there's usually little harm in doing so, and the act of asking the kernel for more permission (read: more virtual memory) is slow. Minor quibble: it's not accurate to call `malloc()` and family in glibc the "operating system's memory allocator." That is the memory allocator for C, and Ruby just so happens to be implemented in C. The glibc allocator will use the system calls `brk()`, `sbrk()` and/or `mmap()` to request memory from the kernel (http://man7.org/linux/man-pages/man2/brk.2.html; http://man7.org/linux/man-pages/man2/brk.2.html; http://man7.org/linux/man-pages/man2/mmap.2.html http://man7.org/linux/man-pages/man2/mmap.2.html). Nothing really changes per the punchline.

stdcli 8y ago
I wrote a bit about this here, with some small GC.stat hacks to manage it a tad, when I talk about how ruby has it's own heap and manages garbage collection lower down in this post: https://ginxh.io/2018-04-18/high-performance-priority-queues-and-ruby/ https://ginxh.io/2018-04-18/high-performance-priority-queues... Even fluentd had this issue with a default heap size lower than what was needed to initialize itself, causing hiccups, until the GC.stat RUBY_GC_HEAP_GROWTH_FACTOR was tuned in the source code. Ruby memory bloat is everywhere. Being familiar with gc.stat and being able to tune ruby applications as you test them is a good habit to have if you develop or work with ruby based tools. Julia Evans, an SRE at Stripe took a sabbatical to work on a ruby memory profiler. Her updates are here: https://jvns.ca/categories/ruby-profiler/ https://jvns.ca/categories/ruby-profiler/ This is a great blog post, it's what I would have wanted to know if I went a bit deeper into this. Good stuff.
microtherion 8y ago
> Having a lot of virtual memory does not "cost" much To some extent, I agree, at least on 64 bit systems (on 32 bit systems, there was the risk of running out of address space). However, the memory in question here is, for the most part, not freshly allocated, but was in use once. This means that it used to be backed by physical memory, and before that backing can be withdrawn, the page has to be written to disk. It seems to me that the best solution would be to call madvise(MADV_FREE) on these regions, in which case they can be unbacked without further ado. I'm somewhat surprised that the memory allocator does not do this itself already.
- the8472 8y ago
  That's what jemalloc does (besides avoiding fragmentation) and one of the reasons why people use it. http://jemalloc.net/jemalloc.3.html#opt.dirty_decay_ms http://jemalloc.net/jemalloc.3.html#opt.dirty_decay_ms
- scott_s 8y ago
  > However, the memory in question here is, for the most part, not freshly allocated, but was in use once. Because the author does not differentiate between virtual and physical memory, I can’t agree with that. It’s quite possible most of the memory “freed” was never in use. In which case, there’s not much benefit. And there is the probable downside that allocation heavy applications will pay a lot more.
  - microtherion 8y ago
    > It’s quite possible most of the memory “freed” was never in use. Given the allocation patterns shown, it would seem to me that if two blocks are still in use, it's fairly likely that the region in between also were in use once (there may be exceptions due to pools etc, but generally memory is parcelled out in a linear fashion). > And there is the probable downside that allocation heavy applications will pay a lot more. What would the cost be? All that would happen is that the free pages are marked as clean. I'm sure that's not entirely free, but bound to be considerably cheaper than paging the page out and in again.
    - scott_s 8y ago
      > What would the cost be? All that would happen is that the free pages are marked as clean. That's a minor page fault. You get an OS-level exception, switch to kernel mode, process the page fault by marking the page as loaded, then switching back to user mode. That is expensive if you do it a lot. > but bound to be considerably cheaper than paging the page out and in again. Yes, of course, a minor page fault is cheaper than a major page fault. But both are more expensive than no page fault, which is what happens if you just never free the page to the OS and there's plenty of available physical memory.
CBLT 8y ago
>it's not accurate to call `malloc()` and family in glibc the "operating system's memory allocator." I watched an interview with Bryan Cantrill some time ago and one of the things he mentioned is that libc is considered part of the operating system for other unixes, and Linux is the only one that decided to redefine operating system to exclude libc.
- pjmlp 8y ago
  As per ISO C, libc is part of the compiler. Whatever else ends up there, like POSIX support, is implementation specific.
  - jchw 8y ago
    I wonder why the ISO C standard is specifying who is responsible for implementing libc. That seems screwy at best. Regardless, afaik OpenBSD and FreeBSD both ship libc as part of the operating system. It makes a lot of sense, really, because libc is bound to need to make syscalls, which differ strongly depending on the kernel, whereas the user interface of libc is pretty much the same across any operating system.
    - hangonhn 8y ago
      Would memory allocation also be hardware dependent and thus vary as well depending on what hardware the OS decides to support?
      - jchw 8y ago
        You mean like devices without MMUs? I think that might be the case but I'm not sure. Under Linux you need a special kernel and libc to handle such devices IIRC.
    - pjmlp 8y ago
      Because not every OS is an UNIX clone and C also targets bare metal.
usr1106 8y ago
The author never made clear if they are measuring virtual memory usage, or physical memory usage. Having a lot of virtual memory does not "cost" much: it's just having permission to use a lot of memory if you so wish. It's possible to have huge chunks of your virtual address space with no physical memory backing them. Physical memory is the amount of physical RAM your process is using. Yes, that remains very unclear. Only physical memory is interesting in the end, address space exhaustion is not on issue on most systems today and large pages are typically not used. The graphs have "virtual", "dirty" and "clean" in them. "clean" normally means that the page in RAM is just a copy of a mass storage. If the kernel runs short of memory, it can use this page for other purposes. No harm done, except that the system gets slower when it needs to page in the same page later. "dirty" normally means that the page in RAM is not backed by mass storage. The kernel must keep it reserved for the current purpose in order not to lose data. If we assume that author's application does not do swapping (it really shouldn't for 230 MB, otherwise the machine is seriously overloaded and really slow) all heap in use is always dirty. The amount of dirty equals usage of physical RAM. I am not sure which Linux tool that can easily report the use of physical RAM for the heap. Would RssAnon from /proc/self/status be a good approximation? It certainly contains the stack, too. But that should not grow a lot unless you have infinite recursion.
FooBarWidget 8y ago
Author here. I measured RSS with swap disabled, so physical memory usage. > Minor quibble: it's not accurate to call `malloc()` and family in glibc the "operating system's memory allocator." I know. I just explained it like that in order to keep the material digestible to a wider audience.
- usr1106 8y ago
  Thank you for the clarification. > I measured RSS with swap disabled, so physical memory usage. That sounds correct to me. More details you could get from /proc/<pid>/smaps. There are a couple of other issues I found confusing in your text. If you intentionally make simplifications, I would recommend to add at least a footnote to indicate so. Linux has only 1 heap. But while the heap is declared to be of a certain maximum size using brk() it does not mean that all of the pages are really in RAM (as before, swapping aside). A page gets only really allocated when it is accessed the first time. And it can get deallocated again using madvise() if the program know that it is no longer needed. malloc_trim() calls madvise(). So you are correct, it does not only move the top of the heap. Probably it was that way many years ago. So in general the Linux heap can be sparsely mapped to RAM. But if you use RSS your measuring takes that into account already. The arenas are a feature of glibc. They are documented (to some degree) in the man pages, so I would not call them magic. Also malloc_info() prints how they are used. It appears to me that arena 0 is on the Linux heap. If a program is multithreaded, glibc will call mmap() to create an additional memory area to be used for the additional arenas. While avoiding mutex contention might be a good thing if the threads are allocating many small blocks, the overhead for arenas might not be justified for rarely allocating bigger areas as the Ruby heap implementation probably does it. So limiting the number of arenas might indeed be beneficial, it's just a typical trade-off one needs to understand. Not that that is easy, but magic is the wrong description IMHO. Let's call the glibc functionality above the system allocator (I'm not sure whether this is the correct name or whether it even has an official name). However, glibc has yet another completely different allocator, the mmap allocator. For allocations bigger than MMAP_THRESHOLD glibc will not use the heap or any of the additional arenas, but just directly make a completely new memory mapping from the Linux kernel. Again these will lazily/sparsely allocated to RAM. The mmap allocator does not use arenas at all. I am not a regular Ruby user, so I don't know whether the Ruby heap uses glibcs's system allocator or the mmap allocator or what mix of both. (Mixing them is transparent to a program.) However, from your description I would guess that it mostly uses the system allocator, otherwise arenas would not make any change and AFAIK malloc_trim() has no effect on the mmap allocator at all. Have you considered either changing the setting of MMAP_THRESHOLD or the Ruby interpreter code such, that the Ruby heap uses only the mmap allocator? It appears to me that having the Ruby heap management on top of the glibc system allocator is just not a good idea. The system allocator tries to be a good compromise for a widely varying spectrum of applications. But the Ruby heap management is one very specific case, partially duplicating the work that the glibc allocator does. I'd guess having the Ruby heap running closer to the operating system should improve things. The mmap allocator of glibc might be an easy way to achieve that. Otherwise the Ruby head should use mmap() directly, because it should know best how it wants to use the memory. But that would be much more difficult to implement of course. P.S. Your visualizer looks impressive. Unfortunately I did not have time to study it in detail. Did you make sure that the code does never access a page that has been mapped but never accessed by Ruby? Because that would dirty the page and increase the RSS.

stdcli 8y ago

I wrote a bit about this here, with some small GC.stat hacks to manage it a tad, when I talk about how ruby has it's own heap and manages garbage collection lower down in this post: https://ginxh.io/2018-04-18/high-performance-priority-queues-and-ruby/ https://ginxh.io/2018-04-18/high-performance-priority-queues... Even fluentd had this issue with a default heap size lower than what was needed to initialize itself, causing hiccups, until the GC.stat RUBY_GC_HEAP_GROWTH_FACTOR was tuned in the source code. Ruby memory bloat is everywhere. Being familiar with gc.stat and being able to tune ruby applications as you test them is a good habit to have if you develop or work with ruby based tools. Julia Evans, an SRE at Stripe took a sabbatical to work on a ruby memory profiler. Her updates are here: https://jvns.ca/categories/ruby-profiler/ https://jvns.ca/categories/ruby-profiler/ This is a great blog post, it's what I would have wanted to know if I went a bit deeper into this. Good stuff.

microtherion 8y ago

> Having a lot of virtual memory does not "cost" much To some extent, I agree, at least on 64 bit systems (on 32 bit systems, there was the risk of running out of address space). However, the memory in question here is, for the most part, not freshly allocated, but was in use once. This means that it used to be backed by physical memory, and before that backing can be withdrawn, the page has to be written to disk. It seems to me that the best solution would be to call madvise(MADV_FREE) on these regions, in which case they can be unbacked without further ado. I'm somewhat surprised that the memory allocator does not do this itself already.

the8472 8y ago
That's what jemalloc does (besides avoiding fragmentation) and one of the reasons why people use it. http://jemalloc.net/jemalloc.3.html#opt.dirty_decay_ms http://jemalloc.net/jemalloc.3.html#opt.dirty_decay_ms
scott_s 8y ago
> However, the memory in question here is, for the most part, not freshly allocated, but was in use once. Because the author does not differentiate between virtual and physical memory, I can’t agree with that. It’s quite possible most of the memory “freed” was never in use. In which case, there’s not much benefit. And there is the probable downside that allocation heavy applications will pay a lot more.
- microtherion 8y ago
  > It’s quite possible most of the memory “freed” was never in use. Given the allocation patterns shown, it would seem to me that if two blocks are still in use, it's fairly likely that the region in between also were in use once (there may be exceptions due to pools etc, but generally memory is parcelled out in a linear fashion). > And there is the probable downside that allocation heavy applications will pay a lot more. What would the cost be? All that would happen is that the free pages are marked as clean. I'm sure that's not entirely free, but bound to be considerably cheaper than paging the page out and in again.
  - scott_s 8y ago
    > What would the cost be? All that would happen is that the free pages are marked as clean. That's a minor page fault. You get an OS-level exception, switch to kernel mode, process the page fault by marking the page as loaded, then switching back to user mode. That is expensive if you do it a lot. > but bound to be considerably cheaper than paging the page out and in again. Yes, of course, a minor page fault is cheaper than a major page fault. But both are more expensive than no page fault, which is what happens if you just never free the page to the OS and there's plenty of available physical memory.

CBLT 8y ago

>it's not accurate to call `malloc()` and family in glibc the "operating system's memory allocator." I watched an interview with Bryan Cantrill some time ago and one of the things he mentioned is that libc is considered part of the operating system for other unixes, and Linux is the only one that decided to redefine operating system to exclude libc.

pjmlp 8y ago
As per ISO C, libc is part of the compiler. Whatever else ends up there, like POSIX support, is implementation specific.
- jchw 8y ago
  I wonder why the ISO C standard is specifying who is responsible for implementing libc. That seems screwy at best. Regardless, afaik OpenBSD and FreeBSD both ship libc as part of the operating system. It makes a lot of sense, really, because libc is bound to need to make syscalls, which differ strongly depending on the kernel, whereas the user interface of libc is pretty much the same across any operating system.
  - hangonhn 8y ago
    Would memory allocation also be hardware dependent and thus vary as well depending on what hardware the OS decides to support?
    - jchw 8y ago
      You mean like devices without MMUs? I think that might be the case but I'm not sure. Under Linux you need a special kernel and libc to handle such devices IIRC.
  - pjmlp 8y ago
    Because not every OS is an UNIX clone and C also targets bare metal.

usr1106 8y ago

The author never made clear if they are measuring virtual memory usage, or physical memory usage. Having a lot of virtual memory does not "cost" much: it's just having permission to use a lot of memory if you so wish. It's possible to have huge chunks of your virtual address space with no physical memory backing them. Physical memory is the amount of physical RAM your process is using. Yes, that remains very unclear. Only physical memory is interesting in the end, address space exhaustion is not on issue on most systems today and large pages are typically not used. The graphs have "virtual", "dirty" and "clean" in them. "clean" normally means that the page in RAM is just a copy of a mass storage. If the kernel runs short of memory, it can use this page for other purposes. No harm done, except that the system gets slower when it needs to page in the same page later. "dirty" normally means that the page in RAM is not backed by mass storage. The kernel must keep it reserved for the current purpose in order not to lose data. If we assume that author's application does not do swapping (it really shouldn't for 230 MB, otherwise the machine is seriously overloaded and really slow) all heap in use is always dirty. The amount of dirty equals usage of physical RAM. I am not sure which Linux tool that can easily report the use of physical RAM for the heap. Would RssAnon from /proc/self/status be a good approximation? It certainly contains the stack, too. But that should not grow a lot unless you have infinite recursion.

FooBarWidget 8y ago

Author here. I measured RSS with swap disabled, so physical memory usage. > Minor quibble: it's not accurate to call `malloc()` and family in glibc the "operating system's memory allocator." I know. I just explained it like that in order to keep the material digestible to a wider audience.

usr1106 8y ago
Thank you for the clarification. > I measured RSS with swap disabled, so physical memory usage. That sounds correct to me. More details you could get from /proc/<pid>/smaps. There are a couple of other issues I found confusing in your text. If you intentionally make simplifications, I would recommend to add at least a footnote to indicate so. Linux has only 1 heap. But while the heap is declared to be of a certain maximum size using brk() it does not mean that all of the pages are really in RAM (as before, swapping aside). A page gets only really allocated when it is accessed the first time. And it can get deallocated again using madvise() if the program know that it is no longer needed. malloc_trim() calls madvise(). So you are correct, it does not only move the top of the heap. Probably it was that way many years ago. So in general the Linux heap can be sparsely mapped to RAM. But if you use RSS your measuring takes that into account already. The arenas are a feature of glibc. They are documented (to some degree) in the man pages, so I would not call them magic. Also malloc_info() prints how they are used. It appears to me that arena 0 is on the Linux heap. If a program is multithreaded, glibc will call mmap() to create an additional memory area to be used for the additional arenas. While avoiding mutex contention might be a good thing if the threads are allocating many small blocks, the overhead for arenas might not be justified for rarely allocating bigger areas as the Ruby heap implementation probably does it. So limiting the number of arenas might indeed be beneficial, it's just a typical trade-off one needs to understand. Not that that is easy, but magic is the wrong description IMHO. Let's call the glibc functionality above the system allocator (I'm not sure whether this is the correct name or whether it even has an official name). However, glibc has yet another completely different allocator, the mmap allocator. For allocations bigger than MMAP_THRESHOLD glibc will not use the heap or any of the additional arenas, but just directly make a completely new memory mapping from the Linux kernel. Again these will lazily/sparsely allocated to RAM. The mmap allocator does not use arenas at all. I am not a regular Ruby user, so I don't know whether the Ruby heap uses glibcs's system allocator or the mmap allocator or what mix of both. (Mixing them is transparent to a program.) However, from your description I would guess that it mostly uses the system allocator, otherwise arenas would not make any change and AFAIK malloc_trim() has no effect on the mmap allocator at all. Have you considered either changing the setting of MMAP_THRESHOLD or the Ruby interpreter code such, that the Ruby heap uses only the mmap allocator? It appears to me that having the Ruby heap management on top of the glibc system allocator is just not a good idea. The system allocator tries to be a good compromise for a widely varying spectrum of applications. But the Ruby heap management is one very specific case, partially duplicating the work that the glibc allocator does. I'd guess having the Ruby heap running closer to the operating system should improve things. The mmap allocator of glibc might be an easy way to achieve that. Otherwise the Ruby head should use mmap() directly, because it should know best how it wants to use the memory. But that would be much more difficult to implement of course. P.S. Your visualizer looks impressive. Unfortunately I did not have time to study it in detail. Did you make sure that the code does never access a page that has been mapped but never accessed by Ruby? Because that would dirty the page and increase the RSS.