3 ms·

brk does grow linearly (a holdover from the days of segmentation). But malloc may well allocate chunks of virtual memory for large allocations. Windows doesn'

brk does grow linearly (a holdover from the days of segmentation). But malloc may well allocate chunks of virtual memory for large allocations. Windows doesn't have a brk equivalent and the underlying heap is fully implemented via VirtualAlloc.

repsilat 8y ago
Fair, but I think this all amounts to "A linear patch of memory is fine primitive to build an allocator or standard library on." The "not faulting on address zero" thing is interesting but not crazy -- only broken programs will break.
- slededit 8y ago
  While you can build it that way it is not at all a good way to do it. You lose all the benefits of paging when handling fragmentation - you basically will never be able to shrink the data segment on most practical programs - and therefore never be able to return memory to the operating system. brk is an ancient holdover from when memory protection was done via segmentation. Paging eliminates the need for this historical misfeature. I'm a bit sad WASM propagated it further into modern systems.
  - repsilat 8y ago
    Isn't paging orthogonal to using brk? Or is "paging" something more specific than "virtual memory is handled by the OS in big chunks"? I guess your allocator can try its best to prevent fragmentation, but you're right that returning memory to the OS isn't going to happen without explicit compaction (which nobody in native code is going to do.) Dunno if anyone ever expects memory use to go down though. (Is less true than it used to be? -- I guess we used to make big swap files and now that's less fashionable.) I guess if it's not too much more of a burden on the OS to make it keep track of a Swiss cheese virtual memory allocation then it is useful.
    - slededit 8y ago
      The OS implements brk under the hood with "swiss cheese virtual memory" so you don't get any real optimizations here. On AMD64 paging is mandatory. It does have a cost but the MMU hardware accelerates this to make it mostly not noticeable. The brk system call places an unnecessary restriction on how the memory is used that is not imposed by the hardware. Back in the 70s when segmentation was a thing and virtual memory hadn't taken hold this would be different - but virtual memory has won.
floatboth 8y ago
Pretty much only glibc malloc still uses brk. GNU being GNU I guess. Modern mallocs like jemalloc use mmap exclusively. FreeBSD straight up did not implement sbrk at all for aarch64 and riscv – very very few things broke because of that (one of the things that did break was, IIRC, GNU Emacs).

repsilat 8y ago

Fair, but I think this all amounts to "A linear patch of memory is fine primitive to build an allocator or standard library on." The "not faulting on address zero" thing is interesting but not crazy -- only broken programs will break.

slededit 8y ago
While you can build it that way it is not at all a good way to do it. You lose all the benefits of paging when handling fragmentation - you basically will never be able to shrink the data segment on most practical programs - and therefore never be able to return memory to the operating system. brk is an ancient holdover from when memory protection was done via segmentation. Paging eliminates the need for this historical misfeature. I'm a bit sad WASM propagated it further into modern systems.
- repsilat 8y ago
  Isn't paging orthogonal to using brk? Or is "paging" something more specific than "virtual memory is handled by the OS in big chunks"? I guess your allocator can try its best to prevent fragmentation, but you're right that returning memory to the OS isn't going to happen without explicit compaction (which nobody in native code is going to do.) Dunno if anyone ever expects memory use to go down though. (Is less true than it used to be? -- I guess we used to make big swap files and now that's less fashionable.) I guess if it's not too much more of a burden on the OS to make it keep track of a Swiss cheese virtual memory allocation then it is useful.
  - slededit 8y ago
    The OS implements brk under the hood with "swiss cheese virtual memory" so you don't get any real optimizations here. On AMD64 paging is mandatory. It does have a cost but the MMU hardware accelerates this to make it mostly not noticeable. The brk system call places an unnecessary restriction on how the memory is used that is not imposed by the hardware. Back in the 70s when segmentation was a thing and virtual memory hadn't taken hold this would be different - but virtual memory has won.

floatboth 8y ago

Pretty much only glibc malloc still uses brk. GNU being GNU I guess. Modern mallocs like jemalloc use mmap exclusively. FreeBSD straight up did not implement sbrk at all for aarch64 and riscv – very very few things broke because of that (one of the things that did break was, IIRC, GNU Emacs).