6 ms·

I thought the memory cost of threads was more of a JVM thing (default 1mb stack) than an OS thing (pages that aren't mapped/resident don't cost much). What's t

I thought the memory cost of threads was more of a JVM thing (default 1mb stack) than an OS thing (pages that aren't mapped/resident don't cost much). What's the cost to the kernel besides a few structs?

pron 6y ago
Once a page is committed, it cannot be uncommitted until the thread dies, because the OS can't be sure how much of the stack is actually used. It cannot even assume that only addresses above sp are used. Also, the granularity is that of a page, which could be significantly larger than a whole stack of some small, "shallow" thread, and we want lots of small threads.
- marvy 6y ago
  How many parked "fibers" do you think a 32-bit JVM might be able to handle once loom is ready for prime time? A thousand? A million? Somewhere in between?
  - pron 6y ago
    When somebody shows up to do a 32-bit port we'll think about it :)
    - marvy 6y ago
      I don't see why you need to wait for that. You already know the memory usage for 64-bit, and you implied elsewhere in this thread that memory usage is the key metric that makes these things cheaper than kernel threads. So you should be able to give an order of magnitude guesstimate even if no 32-bit port is in the works yet. Or no?
      - pron 6y ago
        OK, so let's see. Suppose you have 2GB available for threads, and you want a million threads. This leaves 2KB per thread. Can 2KB suffice for a thread? Sure, for some. Could most threads take up less than 2KB? Probably not. Right now the representation of the stack in RAM is not very efficient, but it could be made much more so. If anyone wants to port to 32 bit and that turns out to be a problem, then I guess improving the stack representation would become a higher priority.
        marvy 6y ago
        Thanks! I think that answers my question. Since kernel threads will likely have 64KB stacks or even more, this is still a huge win, even for 32-bit code I guess.
  - chris_overseas 6y ago
    I doubt 32bit is an issue for this and I'd assume well over a million, if Kotlin's coroutines are anything to go by: https://kotlinlang.org/docs/tutorials/coroutines/coroutines-basic-jvm.html#lets-run-a-lot-of-them https://kotlinlang.org/docs/tutorials/coroutines/coroutines-...
    - marvy 6y ago
      32-bit means you have at most 4 gigs of address space to play with; pron implied that lower memory usage is the key savings of fibers vs threads, so I assume a 32-bit JVM will hit a memory limit a lot sooner.
- deleted 6y ago
  [deleted]
- kgoutham93 6y ago
  Hey Ron, firstly awesome write-up. Can you force yourself to list a few disadvantages of virtual threads over native threads?
  - pron 6y ago
    I wouldn't use virtual threads for long-running, compute-heavy tasks, because even when we expose forced preemption, it might not be as efficient as a kernel preemption. Also, if you run a lot of native (non-Java) code that blocks -- or that upcalls back into Java and then blocks -- then the underlying "carrier" kernel thread will be blocked, and if this happens a lot, it can impact other virtual threads that share the same scheduler. So for compute-heavy jobs and for code that blocks in native or upcalls to Java, the platform threads would be more appropriate.
- jlokier 6y ago
  > the OS can't be sure how much of the stack is actually used. It cannot even assume that only addresses above sp are used A nitpick, and a possible optimization opportunity. If there are any unmasked signal handlers not using SA_ONSTACK, the OS can reasonably assume the thread doesn't care about memory below sp - redzone (redzone is 128 bytes on AMD64) and therefore there would be no harm in reclaiming it.
asdfasgasdgasdg 6y ago
Well, there is a cap on the number of PIDs you can have going, I think. Also, it is my understanding that thread switching causes TLB flushes, which can be expensive. You can also fit several stackless thread contexts within a single page, depending on the size of the context, whereas you'll certainly never get more than one stackful thread into a single page.
- MaxBarraclough 6y ago
  I don't think there should be a TLB flush for a switch between threads, as they share a memory space. It's only necessary for switching between processes. https://stackoverflow.com/a/5440165/ https://stackoverflow.com/a/5440165/
  - asdfasgasdgasdg 6y ago
    Oops, I think you're right and I was wrong.

pron 6y ago

Once a page is committed, it cannot be uncommitted until the thread dies, because the OS can't be sure how much of the stack is actually used. It cannot even assume that only addresses above sp are used. Also, the granularity is that of a page, which could be significantly larger than a whole stack of some small, "shallow" thread, and we want lots of small threads.

marvy 6y ago
How many parked "fibers" do you think a 32-bit JVM might be able to handle once loom is ready for prime time? A thousand? A million? Somewhere in between?
- pron 6y ago
  When somebody shows up to do a 32-bit port we'll think about it :)
  - marvy 6y ago
    I don't see why you need to wait for that. You already know the memory usage for 64-bit, and you implied elsewhere in this thread that memory usage is the key metric that makes these things cheaper than kernel threads. So you should be able to give an order of magnitude guesstimate even if no 32-bit port is in the works yet. Or no?
    - pron 6y ago
      OK, so let's see. Suppose you have 2GB available for threads, and you want a million threads. This leaves 2KB per thread. Can 2KB suffice for a thread? Sure, for some. Could most threads take up less than 2KB? Probably not. Right now the representation of the stack in RAM is not very efficient, but it could be made much more so. If anyone wants to port to 32 bit and that turns out to be a problem, then I guess improving the stack representation would become a higher priority.
      - marvy 6y ago
        Thanks! I think that answers my question. Since kernel threads will likely have 64KB stacks or even more, this is still a huge win, even for 32-bit code I guess.
- chris_overseas 6y ago
  I doubt 32bit is an issue for this and I'd assume well over a million, if Kotlin's coroutines are anything to go by: https://kotlinlang.org/docs/tutorials/coroutines/coroutines-basic-jvm.html#lets-run-a-lot-of-them https://kotlinlang.org/docs/tutorials/coroutines/coroutines-...
  - marvy 6y ago
    32-bit means you have at most 4 gigs of address space to play with; pron implied that lower memory usage is the key savings of fibers vs threads, so I assume a 32-bit JVM will hit a memory limit a lot sooner.
deleted 6y ago
[deleted]
kgoutham93 6y ago
Hey Ron, firstly awesome write-up. Can you force yourself to list a few disadvantages of virtual threads over native threads?
- pron 6y ago
  I wouldn't use virtual threads for long-running, compute-heavy tasks, because even when we expose forced preemption, it might not be as efficient as a kernel preemption. Also, if you run a lot of native (non-Java) code that blocks -- or that upcalls back into Java and then blocks -- then the underlying "carrier" kernel thread will be blocked, and if this happens a lot, it can impact other virtual threads that share the same scheduler. So for compute-heavy jobs and for code that blocks in native or upcalls to Java, the platform threads would be more appropriate.
jlokier 6y ago
> the OS can't be sure how much of the stack is actually used. It cannot even assume that only addresses above sp are used A nitpick, and a possible optimization opportunity. If there are any unmasked signal handlers not using SA_ONSTACK, the OS can reasonably assume the thread doesn't care about memory below sp - redzone (redzone is 128 bytes on AMD64) and therefore there would be no harm in reclaiming it.

asdfasgasdgasdg 6y ago

Well, there is a cap on the number of PIDs you can have going, I think. Also, it is my understanding that thread switching causes TLB flushes, which can be expensive. You can also fit several stackless thread contexts within a single page, depending on the size of the context, whereas you'll certainly never get more than one stackful thread into a single page.

MaxBarraclough 6y ago
I don't think there should be a TLB flush for a switch between threads, as they share a memory space. It's only necessary for switching between processes. https://stackoverflow.com/a/5440165/ https://stackoverflow.com/a/5440165/
- asdfasgasdgasdg 6y ago
  Oops, I think you're right and I was wrong.