4 ms·

One of my biggest irritations is the tiny size of default stacks in 64-bit OSes. They don't really seem to have grown significantly while both physical memory a

One of my biggest irritations is the tiny size of default stacks in 64-bit OSes. They don't really seem to have grown significantly while both physical memory and address space size have grown massively. I write a bunch of algorithms which would most naturally be written recursively, but I have to go out of my way to make sure they have enough stack. Is there a good reason not to give the main thread of every program on a 64-bit OS 1GB of virtual address space?

tempodox 6y ago
On a Unix-like OS you can do ulimit -s <stack-size-in-KiB> At least you only have to set it once per shell instance; put it in your shell init file (bashrc or whatever).
- CJefferson 6y ago
  The problem is, it's hard to distribute libraries which want lots of stack space, because not everyone will do that. In Rust (where threadsafety is easy and cheap), I tend to just spin up a thread of known (large) stack size, and do computations there.
  - tempodox 6y ago
    I see. And the end user might not even know that library is built into the tool they use, or know how to alter the stack size limit on their machine. So I can't think of a better solution either.
- ufo 6y ago
  Can a process increase their own stack limit or do they need to spawn a child process with a higher limit?
ufo 6y ago
The other question I always had is why do we need to have a fixed limit for the stack. Why can't it grow automatically as needed, just like the heap?
- CJefferson 6y ago
  I don't know either -- I understand in 32-bit OSes, as you need continuous memory space and that can be lacking (early Rust experimented with "segmented stacks", where the stack can be split, and took it out as it was too hard to do). However, in 64-bit OSes, just set up a GB of address space for the stack and be done with it.
- chriswarbo 6y ago
  I think this is partly Stockholm syndrome. Having a finite stack will cut off infinite recursion before it consumes the whole memory, and infinite recursion is often an error (e.g. if we get some branching logic wrong, our base case may never be taken). Over time this has become self-reinforcing: - Recursive algorithms may hit stack limits, which makes them less attractive, so programmers are less likely to choose them - Error rates presumably remain pretty constant, so this avoidance decreases the proportion of recursive programs that are deliberately so, and increases the proportion of recursive programs that are accidentally so - Stack limits remain justified, due to how rarely programmers are choosing recursive approaches, and how often stack overflows help us to stop buggy programs I don't think the 'stops buggy programs' argument is particularly compelling, since that could apply to pretty much any restriction (e.g. we could limit RAM usage to 64K to prevent buggy programms allocating too much, those which want more can edit files in a tempfs; processes should run for no longer than 1 minute, to stop buggy programs, those which need more time can spawn children; etc.). I appreciate that low-level languages are operating under some pretty tight constraints (both in what's available to them, and the performance their users expect). What I find very strange is how many high-level scripting languages impose such stack limits, and don't even eliminate tail-calls. The raison d'etre for such languages is to be more ergonomic and powerful than bare-metal languages, and they don't mind sacrificing a lot of performance to do so (using interpreters, boxed values, dynamic dispatch, etc.); yet they're somehow fine with inheriting these weird restrictions from C & co.?
  - Someone 6y ago
    “we could limit RAM usage to 64K to prevent buggy programms allocating too much, those which want more can edit files in a tempfs; processes should run for no longer than 1 minute” In JCL (https://en.wikipedia.org/wiki/Job_Control_Language https://en.wikipedia.org/wiki/Job_Control_Language), you can specify how much CPU time a program can spend, how many pages it can output before the OS kills it, and have to specify what files it reads and writes. That was useful at a time when CPU time, memory and disk space were scarce and expensive, and still would be useful when sandboxing untrusted code.

tempodox 6y ago

On a Unix-like OS you can do ulimit -s <stack-size-in-KiB> At least you only have to set it once per shell instance; put it in your shell init file (bashrc or whatever).

CJefferson 6y ago
The problem is, it's hard to distribute libraries which want lots of stack space, because not everyone will do that. In Rust (where threadsafety is easy and cheap), I tend to just spin up a thread of known (large) stack size, and do computations there.
- tempodox 6y ago
  I see. And the end user might not even know that library is built into the tool they use, or know how to alter the stack size limit on their machine. So I can't think of a better solution either.
ufo 6y ago
Can a process increase their own stack limit or do they need to spawn a child process with a higher limit?

ufo 6y ago

The other question I always had is why do we need to have a fixed limit for the stack. Why can't it grow automatically as needed, just like the heap?

CJefferson 6y ago
I don't know either -- I understand in 32-bit OSes, as you need continuous memory space and that can be lacking (early Rust experimented with "segmented stacks", where the stack can be split, and took it out as it was too hard to do). However, in 64-bit OSes, just set up a GB of address space for the stack and be done with it.
chriswarbo 6y ago
I think this is partly Stockholm syndrome. Having a finite stack will cut off infinite recursion before it consumes the whole memory, and infinite recursion is often an error (e.g. if we get some branching logic wrong, our base case may never be taken). Over time this has become self-reinforcing: - Recursive algorithms may hit stack limits, which makes them less attractive, so programmers are less likely to choose them - Error rates presumably remain pretty constant, so this avoidance decreases the proportion of recursive programs that are deliberately so, and increases the proportion of recursive programs that are accidentally so - Stack limits remain justified, due to how rarely programmers are choosing recursive approaches, and how often stack overflows help us to stop buggy programs I don't think the 'stops buggy programs' argument is particularly compelling, since that could apply to pretty much any restriction (e.g. we could limit RAM usage to 64K to prevent buggy programms allocating too much, those which want more can edit files in a tempfs; processes should run for no longer than 1 minute, to stop buggy programs, those which need more time can spawn children; etc.). I appreciate that low-level languages are operating under some pretty tight constraints (both in what's available to them, and the performance their users expect). What I find very strange is how many high-level scripting languages impose such stack limits, and don't even eliminate tail-calls. The raison d'etre for such languages is to be more ergonomic and powerful than bare-metal languages, and they don't mind sacrificing a lot of performance to do so (using interpreters, boxed values, dynamic dispatch, etc.); yet they're somehow fine with inheriting these weird restrictions from C & co.?
- Someone 6y ago
  “we could limit RAM usage to 64K to prevent buggy programms allocating too much, those which want more can edit files in a tempfs; processes should run for no longer than 1 minute” In JCL (https://en.wikipedia.org/wiki/Job_Control_Language https://en.wikipedia.org/wiki/Job_Control_Language), you can specify how much CPU time a program can spend, how many pages it can output before the OS kills it, and have to specify what files it reads and writes. That was useful at a time when CPU time, memory and disk space were scarce and expensive, and still would be useful when sandboxing untrusted code.