5 ms·

That quote is wrong. Linux can support over a million threads. Of course they can't all be "active" since there isn't more than a few hundred hardware threads,

That quote is wrong. Linux can support over a million threads. Of course they can't all be "active" since there isn't more than a few hundred hardware threads, but that's also true of green threads.

ec109685 4y ago
There's OS level context switching as threads move on and off the CPU that you don't have to pay with the green threads approach.
chrisseaton 4y ago
> Linux can support over a million threads. How many servers have a spare half-terabyte of RAM to do that though? ((512 * 1024 * 1_000_000) / 1024 / 1024 / 1024 = ~500)
- CraigJPerry 4y ago
  In large envs, 512gb ram servers were circa 2015. The servers being racked in those envs today can have > 10Tb and it would be unlikely to rack one with less than 2Tb, that would be a specialised node for some specific workload.
  - chrisseaton 4y ago
    If you have a 2 TB server, which most cloud instances for example are not, that's still 1/4 of your RAM... just for stacks. That can't be an efficient use of RAM.
    - CraigJPerry 4y ago
      Cells in hardware RAM are only used to hold actual data (and some are wasted to alignment). I may claim to have reserved 512gb of ram for stacks, but unless I’m actually using the space (in which case there’s no issue) then physical ram is not being consumed.
      - chrisseaton 4y ago
        Doesn't Java pre-commit stacks? And even if it doesn't... those TLB entries are also not free.
        CraigJPerry 4y ago
        As in write to each requested memory location to guarantee physical allocation? No, that would be really slow. It’s not like heap memory where you probably want to memset it first, with stack you have a stack pointer so you know whats been written by you and what’s random garbage. Java just requests the memory and linux just smirks and always says “yes sure, whatever you need” - it can do that even while the OOM killer is trying to decide which process looks like it has completed the most useful work so far :-) TLB entries dont really come into it (the size isnt an issue from our stack usage but even if it were, we could use larger pages). TLB flushes could be an issue with that many threads floating around but that’s a different story and not really a memory usage so much as a performance and latency issue.
        chrisseaton 4y ago
        > As in write to each requested memory location to guarantee physical allocation? No you use the advisory bits when you map the memory. It does this so it isn't unable to commit more memory while trying to handle an exception. > Java just requests the memory and linux just smirks and always says “yes sure, whatever you need” Linux doesn't ignore even LOCKED does it? And it certainly can't ignore actual writes to pages, which is what for example pretouch_memory does.
        CraigJPerry 4y ago
        >> No you use the advisory bits when you map the memory. Is that really what’s happening when we create another thread? That doesn’t fit with my understanding here. When i create a new thread, i expect its stack to be allocated before the break so mlock wouldn’t come into it. Even on a jvm with Xms smaller than Xmx (i.e. so the jvm could have to request to move the break and grow the heap to accomodate my new threads private jvm stack), I’m expecting you would have to explicitly request this behaviour somehow (because it’s quite unneighbourly and probably counter productive for most apps). I don’t know how you would do that, maybe there’s a call available under the unsafe packages? I’m conscious I’m saying the jvm - i mean hotspot. >> so it isn't unable to commit more memory while trying to handle an exception Again this doesn’t fit with my understanding - you can receive a further OOME while handling an exception and that’s just bad luck. >> Linux doesn't ignore even LOCKED does it? And it certainly can't ignore actual writes to pages Sure but I’m going on the expectation that doesn’t apply here
- akvadrako 4y ago
  First of all, 512GB of RAM isn't that unusual. If you are supporting a million clients, most applications will need a beefy server. If only 0.1% of your clients are active, that's 1000 cores. Secondly, you don't need 512kb per thread. The kernel will only use pages that are actually used, so by default new threads will use 4kb.
  - chrisseaton 4y ago
    I think, but can't prove right now, that Java pre-commits the stack so it has enough head-room to deal with exceptions. Maybe it shouldn't, but it currently does. And the point still-stands - hardware threads are heavier than virtual threads.
    - ec109685 4y ago
      It allocates the entire stack, but it doesn't write into it, so it doesn't take up resident memory until used.
      - chrisseaton 4y ago
        I'm not a HotSpot expert, but my understanding is it puts the stack through add_reserved_region to commit it.
_old_dude_ 4y ago
The problem is not Linux per se, it's the fact that pthread_create [1] reserves 2M for the stack, so the process runs out of memory if you have 1 million threads (1M * 2M == 2T) [1] https://man7.org/linux/man-pages/man3/pthread_create.3.html https://man7.org/linux/man-pages/man3/pthread_create.3.html
- akira2501 4y ago
  Well.. by default. You can set it to the minimum if you like, which is 16384 bytes.
- PaulDavisThe1st 4y ago
  The stack size is configurable ahead of time. It's not fixed at 2M.
- akvadrako 4y ago
  It only reserves virtual memory.
  - cogman10 4y ago
    Which has a real nasty habit of becoming real memory as applications process stuff.

That quote is wrong. Linux can support over a million threads. Of course they can't all be "active" since there isn't more than a few hundred hardware threads, but that's also true of green threads.

ec109685 4y ago

There's OS level context switching as threads move on and off the CPU that you don't have to pay with the green threads approach.

chrisseaton 4y ago

> Linux can support over a million threads. How many servers have a spare half-terabyte of RAM to do that though? ((512 * 1024 * 1_000_000) / 1024 / 1024 / 1024 = ~500)

CraigJPerry 4y ago
In large envs, 512gb ram servers were circa 2015. The servers being racked in those envs today can have > 10Tb and it would be unlikely to rack one with less than 2Tb, that would be a specialised node for some specific workload.
- chrisseaton 4y ago
  If you have a 2 TB server, which most cloud instances for example are not, that's still 1/4 of your RAM... just for stacks. That can't be an efficient use of RAM.
  - CraigJPerry 4y ago
    Cells in hardware RAM are only used to hold actual data (and some are wasted to alignment). I may claim to have reserved 512gb of ram for stacks, but unless I’m actually using the space (in which case there’s no issue) then physical ram is not being consumed.
    - chrisseaton 4y ago
      Doesn't Java pre-commit stacks? And even if it doesn't... those TLB entries are also not free.
      - CraigJPerry 4y ago
        As in write to each requested memory location to guarantee physical allocation? No, that would be really slow. It’s not like heap memory where you probably want to memset it first, with stack you have a stack pointer so you know whats been written by you and what’s random garbage. Java just requests the memory and linux just smirks and always says “yes sure, whatever you need” - it can do that even while the OOM killer is trying to decide which process looks like it has completed the most useful work so far :-) TLB entries dont really come into it (the size isnt an issue from our stack usage but even if it were, we could use larger pages). TLB flushes could be an issue with that many threads floating around but that’s a different story and not really a memory usage so much as a performance and latency issue.
        chrisseaton 4y ago
        > As in write to each requested memory location to guarantee physical allocation? No you use the advisory bits when you map the memory. It does this so it isn't unable to commit more memory while trying to handle an exception. > Java just requests the memory and linux just smirks and always says “yes sure, whatever you need” Linux doesn't ignore even LOCKED does it? And it certainly can't ignore actual writes to pages, which is what for example pretouch_memory does.
        CraigJPerry 4y ago
        >> No you use the advisory bits when you map the memory. Is that really what’s happening when we create another thread? That doesn’t fit with my understanding here. When i create a new thread, i expect its stack to be allocated before the break so mlock wouldn’t come into it. Even on a jvm with Xms smaller than Xmx (i.e. so the jvm could have to request to move the break and grow the heap to accomodate my new threads private jvm stack), I’m expecting you would have to explicitly request this behaviour somehow (because it’s quite unneighbourly and probably counter productive for most apps). I don’t know how you would do that, maybe there’s a call available under the unsafe packages? I’m conscious I’m saying the jvm - i mean hotspot. >> so it isn't unable to commit more memory while trying to handle an exception Again this doesn’t fit with my understanding - you can receive a further OOME while handling an exception and that’s just bad luck. >> Linux doesn't ignore even LOCKED does it? And it certainly can't ignore actual writes to pages Sure but I’m going on the expectation that doesn’t apply here
akvadrako 4y ago
First of all, 512GB of RAM isn't that unusual. If you are supporting a million clients, most applications will need a beefy server. If only 0.1% of your clients are active, that's 1000 cores. Secondly, you don't need 512kb per thread. The kernel will only use pages that are actually used, so by default new threads will use 4kb.
- chrisseaton 4y ago
  I think, but can't prove right now, that Java pre-commits the stack so it has enough head-room to deal with exceptions. Maybe it shouldn't, but it currently does. And the point still-stands - hardware threads are heavier than virtual threads.
  - ec109685 4y ago
    It allocates the entire stack, but it doesn't write into it, so it doesn't take up resident memory until used.
    - chrisseaton 4y ago
      I'm not a HotSpot expert, but my understanding is it puts the stack through add_reserved_region to commit it.

_old_dude_ 4y ago

The problem is not Linux per se, it's the fact that pthread_create [1] reserves 2M for the stack, so the process runs out of memory if you have 1 million threads (1M * 2M == 2T) [1] https://man7.org/linux/man-pages/man3/pthread_create.3.html https://man7.org/linux/man-pages/man3/pthread_create.3.html

akira2501 4y ago
Well.. by default. You can set it to the minimum if you like, which is 16384 bytes.
PaulDavisThe1st 4y ago
The stack size is configurable ahead of time. It's not fixed at 2M.
akvadrako 4y ago
It only reserves virtual memory.
- cogman10 4y ago
  Which has a real nasty habit of becoming real memory as applications process stuff.