4 ms·

‪Main reason for me to use async instead of threads is when I have a very high number of concurrent connections, that don’t do much in terms of CPU, but will re

‪Main reason for me to use async instead of threads is when I have a very high number of concurrent connections, that don’t do much in terms of CPU, but will require a lot of RAM because of the minimum stack size per thread. If I don’t need that, then threads are fine. ‬

woooooo 4y ago
Does Rust have a high minimum stack per thread? I know managed languages like Java do, but for Rust couldn't it be a page or two of actually-mapped memory? Last I checked, the kernel allocates a bunch of address space per thread but it doesn't consume actual memory unless you touch it and page fault.
- noctune 4y ago
  It seems pretty likely that, at some point in the lifetime of a thread, it will consume a lot of the stack and thus touch those pages. And there isn't a mechanism for deallocating those pages again, right? Tying your memory consumption heavily to the max stack depth also seems like it would be difficult to reason about IMO.
  - convolvatron 4y ago
    for me this is the big point. stack-based memory allocation works great as long as your usage and sharing follows the control flow. but when it doesn't, rust really leaves you in a bad spot and you get to play with: - Arcing everything - using a fixed-sized ancillary array and passing indices - creating threads that correspond to the lifetimes you care about - fighting your way through adding explicit lifetime allocations and - delaying reclamation of _everything_ above it on the stack ... and while workable, none of those things are particularly nice
    - insanitybit 4y ago
      IDK, it's pretty easy to just Arc something if it has to be shared across threads. That's what you do in other languages. It's rare that I care to deal with the thread lifetimes themselves.
- ilammy 4y ago
  Rust defaults loosely follow platform’s libc defaults for improved interoperation. On major platforms the default stack size is 2 megabytes of mapped address space. You can ask for less, but it gotta be explicit for every spawned thread. That’s mapped space, so depending on the OS it could be more or less friendly to its memory subsystem. Linux won’t bother finding you actual resident pages until you do need them. But if the stack drops back down, it will be swapped out as any other dirty memory. Rust does not do any MADV_DONTNEED magic behind your back to aid with reclaiming memory.
- conradludgate 4y ago
  As far as I'm aware, the stack size per thread typically doesn't actually matter. It's virtual memory until each page is written to
  - karavelov 4y ago
    It matters actually, if your each tasks use 10KiB stack for execution, and you have 1000 tasks, with threads that will consume 12MiB just for the stacks (+2KiB per task due to page alignment). If you use async runtime as Tokio the stack is un-rolled on each suspend point. So if you have 16 worker threads, that means the stacks will consume 192 KiB.
    - conradludgate 4y ago
      This is true. After I wrote my initial reply I did some measurements. After spawning 4096 threads on my M1 mac, I saw 80MB of reported memory usage. The equivalent tasks in tokio after spawning 100,000 of them reached only 100MB memory. This is roughly 20x less memory. One common thought I see a lot though is that each thread uses 1MB (or more) in stack space alone. This just isn't true with modern memory paging

woooooo 4y ago

Does Rust have a high minimum stack per thread? I know managed languages like Java do, but for Rust couldn't it be a page or two of actually-mapped memory? Last I checked, the kernel allocates a bunch of address space per thread but it doesn't consume actual memory unless you touch it and page fault.

noctune 4y ago
It seems pretty likely that, at some point in the lifetime of a thread, it will consume a lot of the stack and thus touch those pages. And there isn't a mechanism for deallocating those pages again, right? Tying your memory consumption heavily to the max stack depth also seems like it would be difficult to reason about IMO.
- convolvatron 4y ago
  for me this is the big point. stack-based memory allocation works great as long as your usage and sharing follows the control flow. but when it doesn't, rust really leaves you in a bad spot and you get to play with: - Arcing everything - using a fixed-sized ancillary array and passing indices - creating threads that correspond to the lifetimes you care about - fighting your way through adding explicit lifetime allocations and - delaying reclamation of _everything_ above it on the stack ... and while workable, none of those things are particularly nice
  - insanitybit 4y ago
    IDK, it's pretty easy to just Arc something if it has to be shared across threads. That's what you do in other languages. It's rare that I care to deal with the thread lifetimes themselves.
ilammy 4y ago
Rust defaults loosely follow platform’s libc defaults for improved interoperation. On major platforms the default stack size is 2 megabytes of mapped address space. You can ask for less, but it gotta be explicit for every spawned thread. That’s mapped space, so depending on the OS it could be more or less friendly to its memory subsystem. Linux won’t bother finding you actual resident pages until you do need them. But if the stack drops back down, it will be swapped out as any other dirty memory. Rust does not do any MADV_DONTNEED magic behind your back to aid with reclaiming memory.
conradludgate 4y ago
As far as I'm aware, the stack size per thread typically doesn't actually matter. It's virtual memory until each page is written to
- karavelov 4y ago
  It matters actually, if your each tasks use 10KiB stack for execution, and you have 1000 tasks, with threads that will consume 12MiB just for the stacks (+2KiB per task due to page alignment). If you use async runtime as Tokio the stack is un-rolled on each suspend point. So if you have 16 worker threads, that means the stacks will consume 192 KiB.
  - conradludgate 4y ago
    This is true. After I wrote my initial reply I did some measurements. After spawning 4096 threads on my M1 mac, I saw 80MB of reported memory usage. The equivalent tasks in tokio after spawning 100,000 of them reached only 100MB memory. This is roughly 20x less memory. One common thought I see a lot though is that each thread uses 1MB (or more) in stack space alone. This just isn't true with modern memory paging