4 ms·

It's funny the parallel of "detect when the mapped stack region is exhausted, and map the next page" and "detect when the stack region is/will exhaust, and rese

It's funny the parallel of "detect when the mapped stack region is exhausted, and map the next page" and "detect when the stack region is/will exhaust, and reserve/allocate another region"

sunshowers 2y ago
Yeah, I was thinking about that myself. This post suggests that actually growing the stack automatically may just be very difficult with standard POSIX APIs: https://news.ycombinator.com/item?id=36020073 https://news.ycombinator.com/item?id=36020073 The Rust runtime installs a SIGSEGV handler but it's purely informative, and it can be overridden without affecting correctness.
- dwattttt 2y ago
  I took a brief look into Windows public API, even with a tight coupling between compiler/toolchain & OS they don't offer a way to increase the region for a stack after a thread has been created.
  - ynik 2y ago
    But if you're on 64-bit; you can just create the threads with a huge stack size limit (e.g. 1GB) and let the OS handle automatically growing the actual stack size. No need to reinvent the wheel. Stack size is the one area where Windows does have something like "overcommit" by default, as you can separately configure the reserved and commit sizes of the stack: https://learn.microsoft.com/en-us/windows/win32/procthread/thread-stack-size https://learn.microsoft.com/en-us/windows/win32/procthread/t...
    - SkiFire13 2y ago
      This just delays the problem though
    - sunshowers 2y ago
      Yes, this approach works on other platforms too. However I believe that this will not deallocate stack space, while stacker will.
- fch42 2y ago
  You can only grow the stack automatically if you "prereserve" the virtual address space for it. Then it becomes just a question of "committing" the memory at time of need. Default options to mmap() should actually give you this (the OS finds/fills the pages for you on first pagefault), but Solaris/Linux differ on overcommit behaviour. To "grow" a stack at runtime, if the virtual address space to just add mappings in the "right" place is unavailable (already used for something else, say), it means a stack switch. That isn't entirely impossible. But if done anywhere else but in the "base thread initialisation" (as seen in that example) it becomes complicated and hazardous. It's more like "coroutines" then, your current function "spawns" a func on a new callstack and on return, undoes that and cleans up. One can imagine things like thread-local signal handlers to "simulate" this transparently but that would become more of a piece of programming performance art. If there ever were an "unbelievable unix underbelly programming contest" (UUUPC) stuff like that might well make it.

It's funny the parallel of "detect when the mapped stack region is exhausted, and map the next page" and "detect when the stack region is/will exhaust, and reserve/allocate another region"

sunshowers 2y ago

Yeah, I was thinking about that myself. This post suggests that actually growing the stack automatically may just be very difficult with standard POSIX APIs: https://news.ycombinator.com/item?id=36020073 https://news.ycombinator.com/item?id=36020073 The Rust runtime installs a SIGSEGV handler but it's purely informative, and it can be overridden without affecting correctness.

dwattttt 2y ago
I took a brief look into Windows public API, even with a tight coupling between compiler/toolchain & OS they don't offer a way to increase the region for a stack after a thread has been created.
- ynik 2y ago
  But if you're on 64-bit; you can just create the threads with a huge stack size limit (e.g. 1GB) and let the OS handle automatically growing the actual stack size. No need to reinvent the wheel. Stack size is the one area where Windows does have something like "overcommit" by default, as you can separately configure the reserved and commit sizes of the stack: https://learn.microsoft.com/en-us/windows/win32/procthread/thread-stack-size https://learn.microsoft.com/en-us/windows/win32/procthread/t...
  - SkiFire13 2y ago
    This just delays the problem though
  - sunshowers 2y ago
    Yes, this approach works on other platforms too. However I believe that this will not deallocate stack space, while stacker will.
fch42 2y ago
You can only grow the stack automatically if you "prereserve" the virtual address space for it. Then it becomes just a question of "committing" the memory at time of need. Default options to mmap() should actually give you this (the OS finds/fills the pages for you on first pagefault), but Solaris/Linux differ on overcommit behaviour. To "grow" a stack at runtime, if the virtual address space to just add mappings in the "right" place is unavailable (already used for something else, say), it means a stack switch. That isn't entirely impossible. But if done anywhere else but in the "base thread initialisation" (as seen in that example) it becomes complicated and hazardous. It's more like "coroutines" then, your current function "spawns" a func on a new callstack and on return, undoes that and cleans up. One can imagine things like thread-local signal handlers to "simulate" this transparently but that would become more of a piece of programming performance art. If there ever were an "unbelievable unix underbelly programming contest" (UUUPC) stuff like that might well make it.