4 ms·

100k+ concurrent threads is a realistic starting point for M:N threading. As long as native threads require a separate preallocated stack that never shrinks the

100k+ concurrent threads is a realistic starting point for M:N threading. As long as native threads require a separate preallocated stack that never shrinks there will always be a mismatch compared to the capabilities you get running thread per core event based code. Making M:N threads a first class POSIX feature is a good starting point as long as it also allows you to express the intended locality of groups of threads. Locality isn't optional anymore. The slowdowns he reports also map to scale up issues you see when you don't think about keeping frequently accessed mutable data local and shared nothing to an OS thread. M:N threads are just an abstraction around a stack and registers that frees you from having to bind up a bunch of related state into a series of objects and maps. It's awesome, I want it, but nothing is quite there yet. At least not if you want something like Java or C/C++ performance. Work stealing across cores is really nice, but I am not sure it is always the behavior you want. A programmer can tell when migrating a task across cores is going to hurt more than waiting a little longer to get to it. I also think any discussion of M:N threads or multi-threading is also incomplete if it doesn't also include garbage collection as a means of passing data between threads. Right now we have languages that are garbage collected and treat native memory as a second class citizen and non-GC languages that make multi-threading a headache. GC is great for multi-threading, but not so great for hundreds of gigabytes of variable lifetime objects. Even if the GC can handle it, GC still has unacceptable 2x space overhead. I don't want much :-) The youtube video doesn't load for me :-(

viraptor 13y ago
With the nice tuning and code standards you don't need the gigabytes of variable lifetime objects. Of course the whole environment needs to be available in the right way, but for example it's not uncommon in erlang to see thousands of processes that have the memory size adjusted enough to never GC and finish quickly. Then you collect the whole process, rather than separate objects. Immutability also helps here. Not many languages can afford doing that and of course there may be lots of features that prevent similar behaviour.
- rdtsc 13y ago
  In Erlang GC is boringly easy. It has separate heaps for processes and processes are isolated. The price to pay is copying some data between processes and handling immutability. But boring is good. They manage a pauseless and concurrent GC without a sweat.
  - pron 13y ago
    Well, yes and no. Erlang's GC is simple, but incomplete. Shared data structures like ETS aren't GC-ed, and there are many data structures that you simply can't implement in Erlang because of the way that GC works. For example, you can't implement Java's concurrent hash-map in Erlang. Erlang relies on C code whenever it needs shared data structures. So the simple and effective GC comes at the cost of a limited programming model. For Erlang, it's the right tradeoff, but not for every environment.
rdtsc 13y ago
The size of the OS threads with their stacks is what I am worried about as well. It is interesting the author hints that: > It drops further when asked to allocate 8MiB stacks like C is doing Memory, just like CPU is constrained and costs money especially if provisioned in the cloud. If I have 2G of memory available I can spawn 250 threads and keep them around without swapping. EDIT: actually scratch the above, _wmd pointed out that stack memory is just like virtual memory, it is allocated but brought into physical memory as needed. This changes how code is written. It is not just an internal detail! Now you cannot think "one-connection = one-task" now it is about locks, queues and mapping control blocks to connections. That is the biggest loss. > GC is great for multi-threading, but not so great for hundreds of gigabytes of variable lifetime objects. Even if the GC can handle it, GC still has unacceptable 2x space overhead Good point on GC. I see the main issues in a large concurrent system (and presumably Rust wants to be a good tool for concurrent systems) is non-blocking GC. A slow GC might be acceptable, but if one errant task allocated memory in certain pattern and GC has to lock all the tasks out to run, that could be a serious issue. Responsiveness (liveliness) of a concurrent system is something that is often forgotten and everyone wants to talk about how fast one can transpose a matrix. Now a concurrent GC is possible, Azul built one, for Java, it is very interesting how it work. I enjoy reading their whitepapers: http://www.azulsystems.com/resources/whitepapers http://www.azulsystems.com/resources/whitepapers
- _wmd 13y ago
  Just highlighting a small, but common miscomprehension (that on one project, caused a PM to mail everyone at 2am about a HUGE nonexistent memory leak).. Thread stacks are virtual memory like everywhere else, so in reality an "8mb" stack means "8mb maximum size". Sure, they won't shrink once pages are faulted in to back them, but in the average application, especially on 64bit, this should never be a problem If for the lifetime of your thread, its stack only ever grew to 32kb, then the OS will only have allocated 32kb of memory to back it.
  - arielweisberg 13y ago
    You won't hit the worst case, but the common case is that threads will burst stack usage during task execution. If you have 100k threads your effective stack usage will be the high watermark. It varies by workload, but for mine that is between 128 and 512 kilobytes per thread. If you churn all your threads no problem. If you are hosting 100k persistent connections I suspect it would detract from available memory.
wmf 13y ago
A little while back Rust decided to drop segmented stacks and use normal stacks (and basically give up on 32-bit), so now they're deciding between M:N with full-size stacks or 1:1 with full-size stacks. The dominoes are falling.
- qznc 13y ago
  In a year Rust is like C with a better type system and ML syntax. Not that interesting as a language anymore, but more useful in practice.
  - riffraff 13y ago
    I thought the same, piling up on the removal of typestate which was the weirdest/most interesting thing in rust.
  - zem 13y ago
    "a better type system" glosses over a lot, though. if all rust can do is bring linear typing and algebraic datatypes to c that will be a significant achievement.
  - pcwalton 13y ago
    > Not that interesting as a language anymore, but more useful in practice. I think Rust is still pretty interesting. The borrow checker is extremely intricate and is completely unlike anything seen in any industry language so far.
  - nwmcsween 13y ago
    I wish there was something like C but with a better type system, I don't think rust will fill that gap. There are still too many inconsistencies between abstractions with overhead and without making it a C++ replacement if anything.
    - pcwalton 13y ago
      > There are still too many inconsistencies between abstractions with overhead and without making it a C++ replacement if anything. For example?
pcwalton 13y ago
> Right now we have languages that are garbage collected and treat native memory as a second class citizen and non-GC languages that make multi-threading a headache. I think Rust did a good job of making a non-garbage-collected language make memory management less of a headache in a multithreaded scenario (though I'm biased of course). In fact, I think it works even better than in languages with global GC. By default you transfer memory between threads ("don't communicate by sharing memory, share memory by communicating"), and the compiler enforces that you can't touch memory after you've given it away. You can also share read-only data (Arc), and if you do that then the compiler will make sure you can't mutate it and race. If you want to use locks, you can use those too (MutexArc), and the compiler will ensure you take and release locks properly. No tracing garbage collection in sight, and no fiddling with race detectors at runtime—the compiler does the work for you.

viraptor 13y ago

With the nice tuning and code standards you don't need the gigabytes of variable lifetime objects. Of course the whole environment needs to be available in the right way, but for example it's not uncommon in erlang to see thousands of processes that have the memory size adjusted enough to never GC and finish quickly. Then you collect the whole process, rather than separate objects. Immutability also helps here. Not many languages can afford doing that and of course there may be lots of features that prevent similar behaviour.

rdtsc 13y ago
In Erlang GC is boringly easy. It has separate heaps for processes and processes are isolated. The price to pay is copying some data between processes and handling immutability. But boring is good. They manage a pauseless and concurrent GC without a sweat.
- pron 13y ago
  Well, yes and no. Erlang's GC is simple, but incomplete. Shared data structures like ETS aren't GC-ed, and there are many data structures that you simply can't implement in Erlang because of the way that GC works. For example, you can't implement Java's concurrent hash-map in Erlang. Erlang relies on C code whenever it needs shared data structures. So the simple and effective GC comes at the cost of a limited programming model. For Erlang, it's the right tradeoff, but not for every environment.

rdtsc 13y ago

The size of the OS threads with their stacks is what I am worried about as well. It is interesting the author hints that: > It drops further when asked to allocate 8MiB stacks like C is doing Memory, just like CPU is constrained and costs money especially if provisioned in the cloud. If I have 2G of memory available I can spawn 250 threads and keep them around without swapping. EDIT: actually scratch the above, _wmd pointed out that stack memory is just like virtual memory, it is allocated but brought into physical memory as needed. This changes how code is written. It is not just an internal detail! Now you cannot think "one-connection = one-task" now it is about locks, queues and mapping control blocks to connections. That is the biggest loss. > GC is great for multi-threading, but not so great for hundreds of gigabytes of variable lifetime objects. Even if the GC can handle it, GC still has unacceptable 2x space overhead Good point on GC. I see the main issues in a large concurrent system (and presumably Rust wants to be a good tool for concurrent systems) is non-blocking GC. A slow GC might be acceptable, but if one errant task allocated memory in certain pattern and GC has to lock all the tasks out to run, that could be a serious issue. Responsiveness (liveliness) of a concurrent system is something that is often forgotten and everyone wants to talk about how fast one can transpose a matrix. Now a concurrent GC is possible, Azul built one, for Java, it is very interesting how it work. I enjoy reading their whitepapers: http://www.azulsystems.com/resources/whitepapers http://www.azulsystems.com/resources/whitepapers

_wmd 13y ago
Just highlighting a small, but common miscomprehension (that on one project, caused a PM to mail everyone at 2am about a HUGE nonexistent memory leak).. Thread stacks are virtual memory like everywhere else, so in reality an "8mb" stack means "8mb maximum size". Sure, they won't shrink once pages are faulted in to back them, but in the average application, especially on 64bit, this should never be a problem If for the lifetime of your thread, its stack only ever grew to 32kb, then the OS will only have allocated 32kb of memory to back it.
- arielweisberg 13y ago
  You won't hit the worst case, but the common case is that threads will burst stack usage during task execution. If you have 100k threads your effective stack usage will be the high watermark. It varies by workload, but for mine that is between 128 and 512 kilobytes per thread. If you churn all your threads no problem. If you are hosting 100k persistent connections I suspect it would detract from available memory.

wmf 13y ago

A little while back Rust decided to drop segmented stacks and use normal stacks (and basically give up on 32-bit), so now they're deciding between M:N with full-size stacks or 1:1 with full-size stacks. The dominoes are falling.

qznc 13y ago
In a year Rust is like C with a better type system and ML syntax. Not that interesting as a language anymore, but more useful in practice.
- riffraff 13y ago
  I thought the same, piling up on the removal of typestate which was the weirdest/most interesting thing in rust.
- zem 13y ago
  "a better type system" glosses over a lot, though. if all rust can do is bring linear typing and algebraic datatypes to c that will be a significant achievement.
- pcwalton 13y ago
  > Not that interesting as a language anymore, but more useful in practice. I think Rust is still pretty interesting. The borrow checker is extremely intricate and is completely unlike anything seen in any industry language so far.
- nwmcsween 13y ago
  I wish there was something like C but with a better type system, I don't think rust will fill that gap. There are still too many inconsistencies between abstractions with overhead and without making it a C++ replacement if anything.
  - pcwalton 13y ago
    > There are still too many inconsistencies between abstractions with overhead and without making it a C++ replacement if anything. For example?

pcwalton 13y ago

> Right now we have languages that are garbage collected and treat native memory as a second class citizen and non-GC languages that make multi-threading a headache. I think Rust did a good job of making a non-garbage-collected language make memory management less of a headache in a multithreaded scenario (though I'm biased of course). In fact, I think it works even better than in languages with global GC. By default you transfer memory between threads ("don't communicate by sharing memory, share memory by communicating"), and the compiler enforces that you can't touch memory after you've given it away. You can also share read-only data (Arc), and if you do that then the compiler will make sure you can't mutate it and race. If you want to use locks, you can use those too (MutexArc), and the compiler will ensure you take and release locks properly. No tracing garbage collection in sight, and no fiddling with race detectors at runtime—the compiler does the work for you.