5 ms·

> There is no way for your operating system to know exactly how much stack space a thread will need so it allocates an amount on the order of around a megabyte.

> There is no way for your operating system to know exactly how much stack space a thread will need so it allocates an amount on the order of around a megabyte. You only have around a bakers dozen gigabytes of RAM, so you can only make give or take 10,000 threads. That's not true at all. The megabyte(s) of stack space is virtual, spawning a thread only requires a few kilobytes of memory initially, and is a lot cheaper than many realize.

taspeotis 4y ago
It's also not true when ... you tell it how much you need. Windows lets you specify a stack size [1]. [1] https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createthread https://docs.microsoft.com/en-us/windows/win32/api/processth...
- maeln 4y ago
  Fun fact, you can declare a stack size at compile time (something like `cl ... -stack:0x5000000,0x5000000 ...`) which let you create insane stack size so that you can fit your all your software in the stack. Useful when you don't want to had an allocator when making tiny tiny executable files :D
SemanticStrengh 4y ago
The main cost of spawning a thread is simply the time it takes to spawn.
- latch 4y ago
  What are you trying to say? This sounds like a tautology to me.
  - MikeTheGreat 4y ago
    I think they meant that the main cost of a new thread that you'd like to create is the time it takes to spawn (i.e., the time it takes to set up the new thread and get it ready to run) and NOT the memory that gets allocated for it, nor the operating system overhead of keeping it around afterwards, etc, etc. (I'm not agreeing or disagreeing; I'm only trying to helpfully clarify)
  - SemanticStrengh 4y ago
    This is not a tautology.
  - kevincox 4y ago
    They are saying that most of the cost is startup cost, not running overhead.
EdwardDiego 4y ago
In Java, it defaults to 1MB on platforms other than Windows. I've seen people OOM themselves by setting the default thread stack size massively high - they had likely gotten confused with the args (they'd used -Xss when I think they meant -Xms) and given their threads 1GiB of stack each. I can't speak to how Java manages stack size between itself and the OS though, I'm sure there's deep magic there.
emccue 4y ago
Here is the part where I admit that I am an idiot and don't actually know. All I do know is that in Java I cannot create this many threads and in the talks given about Project Loom this is given as the explanation as to why. https://www.youtube.com/watch?v=EO9oMiL1fFo https://www.youtube.com/watch?v=EO9oMiL1fFo @ ~4:00 Is it maybe not the actual spawning but some "switch" that flips once you start using the thread? I'll update the post or leave a comment with a correction once I understand exactly what you mean
- vlovich123 4y ago
  With Fibers you can implement user-space switching between virtual threads. For example, if a thread is going into a top-level select, switch it to a small stack putting the original large stack into a pool. On wake resume switch back to a larger stack. That’s basically what Project Loom is doing I suspect.
  - emccue 4y ago
    No, I understand how Loom limits the amount of stuff on the stack. Its just I thought threads always had those large stacks. Based on what the top person said, I'm thinking that they don't get a large stacks until they cross some threshold of use, but I want to make sure.
    - vlovich123 4y ago
      Stacks are allocated statically upfront. However, you can change the size the kernel allocates, you can point them to use a custom memory space instead of the default one and you can change the stack you're running on. Also, like all memory pages, stack space isn't actually allocated until you modify it so if your stack depth doesn't get crazy large/you're not storing large objects on the stack, it's possible your 16 MB stack is only using 1 MB or so of real RAM.
- hddqsb 4y ago
  The explanation in the video is good, here is a transcription for convenience (emphasis added): > OpenJDK implements threads as thin wrappers around the OS thread [...] OS threads are expensive, by which I mean that we cannot have too many of them. Maybe a few thousand active ones, maybe ten thousand active ones, but that's about it. And the reason for that is that the OS doesn't know the various languages, and the most costly component of a thread is its stack, and the operating system doesn't know how the various languages it might support makes use of the stack. So it must address the worst case and allocate a very large stack. It's usually on a megabyte scale. And even though we use virtual memory and not all of it is committed to physical memory, once we touch a page it becomes committed – it can't be uncommitted again because the OS doesn't know how the stack is used. And in any event, all the operations are done at a page granularity which is 4k, it's also not that small. There may also be arbitrary OS-specific limits. For example on Linux there is system-wide limit on the number of threads, controlled by /proc/sys/kernel/threads-max (docs: https://www.kernel.org/doc/html/latest/admin-guide/sysctl/kernel.html#threads-max https://www.kernel.org/doc/html/latest/admin-guide/sysctl/ke..., see also: https://stackoverflow.com/a/8849114 https://stackoverflow.com/a/8849114). On my system it's ~60,000 (with 8G RAM).
torginus 4y ago
But does Loom reclaim allocated stack space once the thread's stack shrinks?
- Skinney 4y ago
  I believe so, yes.
- raducu 4y ago
  > But does Loom reclaim allocated stack space once the thread's stack shrinks? Could you elaborate why you asked this? Is it because it's not done already? I'm genuinely curious.

taspeotis 4y ago

It's also not true when ... you tell it how much you need. Windows lets you specify a stack size [1]. [1] https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createthread https://docs.microsoft.com/en-us/windows/win32/api/processth...

maeln 4y ago
Fun fact, you can declare a stack size at compile time (something like `cl ... -stack:0x5000000,0x5000000 ...`) which let you create insane stack size so that you can fit your all your software in the stack. Useful when you don't want to had an allocator when making tiny tiny executable files :D

SemanticStrengh 4y ago

The main cost of spawning a thread is simply the time it takes to spawn.

latch 4y ago
What are you trying to say? This sounds like a tautology to me.
- MikeTheGreat 4y ago
  I think they meant that the main cost of a new thread that you'd like to create is the time it takes to spawn (i.e., the time it takes to set up the new thread and get it ready to run) and NOT the memory that gets allocated for it, nor the operating system overhead of keeping it around afterwards, etc, etc. (I'm not agreeing or disagreeing; I'm only trying to helpfully clarify)
- SemanticStrengh 4y ago
  This is not a tautology.
- kevincox 4y ago
  They are saying that most of the cost is startup cost, not running overhead.

EdwardDiego 4y ago

In Java, it defaults to 1MB on platforms other than Windows. I've seen people OOM themselves by setting the default thread stack size massively high - they had likely gotten confused with the args (they'd used -Xss when I think they meant -Xms) and given their threads 1GiB of stack each. I can't speak to how Java manages stack size between itself and the OS though, I'm sure there's deep magic there.

emccue 4y ago

Here is the part where I admit that I am an idiot and don't actually know. All I do know is that in Java I cannot create this many threads and in the talks given about Project Loom this is given as the explanation as to why. https://www.youtube.com/watch?v=EO9oMiL1fFo https://www.youtube.com/watch?v=EO9oMiL1fFo @ ~4:00 Is it maybe not the actual spawning but some "switch" that flips once you start using the thread? I'll update the post or leave a comment with a correction once I understand exactly what you mean

vlovich123 4y ago
With Fibers you can implement user-space switching between virtual threads. For example, if a thread is going into a top-level select, switch it to a small stack putting the original large stack into a pool. On wake resume switch back to a larger stack. That’s basically what Project Loom is doing I suspect.
- emccue 4y ago
  No, I understand how Loom limits the amount of stuff on the stack. Its just I thought threads always had those large stacks. Based on what the top person said, I'm thinking that they don't get a large stacks until they cross some threshold of use, but I want to make sure.
  - vlovich123 4y ago
    Stacks are allocated statically upfront. However, you can change the size the kernel allocates, you can point them to use a custom memory space instead of the default one and you can change the stack you're running on. Also, like all memory pages, stack space isn't actually allocated until you modify it so if your stack depth doesn't get crazy large/you're not storing large objects on the stack, it's possible your 16 MB stack is only using 1 MB or so of real RAM.
hddqsb 4y ago
The explanation in the video is good, here is a transcription for convenience (emphasis added): > OpenJDK implements threads as thin wrappers around the OS thread [...] OS threads are expensive, by which I mean that we cannot have too many of them. Maybe a few thousand active ones, maybe ten thousand active ones, but that's about it. And the reason for that is that the OS doesn't know the various languages, and the most costly component of a thread is its stack, and the operating system doesn't know how the various languages it might support makes use of the stack. So it must address the worst case and allocate a very large stack. It's usually on a megabyte scale. And even though we use virtual memory and not all of it is committed to physical memory, once we touch a page it becomes committed – it can't be uncommitted again because the OS doesn't know how the stack is used. And in any event, all the operations are done at a page granularity which is 4k, it's also not that small. There may also be arbitrary OS-specific limits. For example on Linux there is system-wide limit on the number of threads, controlled by /proc/sys/kernel/threads-max (docs: https://www.kernel.org/doc/html/latest/admin-guide/sysctl/kernel.html#threads-max https://www.kernel.org/doc/html/latest/admin-guide/sysctl/ke..., see also: https://stackoverflow.com/a/8849114 https://stackoverflow.com/a/8849114). On my system it's ~60,000 (with 8G RAM).

torginus 4y ago

But does Loom reclaim allocated stack space once the thread's stack shrinks?

Skinney 4y ago
I believe so, yes.
raducu 4y ago
> But does Loom reclaim allocated stack space once the thread's stack shrinks? Could you elaborate why you asked this? Is it because it's not done already? I'm genuinely curious.