6 ms·

Async has never been about speed. Cooperative multitasking doesn't magically gzip your instruction pipeline or something (actually CM is suboptimal). It's just

by plq 12y ago

Async has never been about speed. Cooperative multitasking doesn't magically gzip your instruction pipeline or something (actually CM is suboptimal). It's just the only tool for those of us who want to avoid giving the keys to arguably the most important subsystem in the kernel to agents outside our control: The scheduler. Ever tried to ssh into a one-thread-per-connection setup under heavy load? Assuming you managed to log in, it'll be very very difficult to get htop to execute when it's competing with 2k other processes for cpu time. So, spawning a thread for every incoming connection request is a bad idea. But what about spawning a thread for every outgoing database connection? The memory overhead of a thread compared to a tcp socket may be significant, but it's nothing compared to a database connection and all the other resources tied up serve that one database query. And best of all, you can refuse to setup a database connection without upsetting your user and e.g. serve from cache instead, but you can't refuse an incoming connection request. According to the Postgres guys [1]: "A formula which has held up pretty well across a lot of benchmarks for years is that for optimal throughput the number of active connections should be somewhere near ((core_count * 2) + effective_spindle_count)" Assuming you got two tablespaces on two raid 1 arrays on a 16 core machine, that's 16 * 2 + 2 = 34 threads plus the event loop thread, maximum. According to those numbers, there doesn't seem to be much stopping you from running your data access logic within a thread. [1]: https://wiki.postgresql.org/wiki/Number_Of_Database_Connections https://wiki.postgresql.org/wiki/Number_Of_Database_Connecti...

thrownaway2424 12y ago
"Ever tried to ssh into a one-thread-per-connection setup under heavy load? Assuming you managed to log in, it'll be very very difficult to get htop to execute when it's competing with 2k other processes for cpu time." Nothing about your comment is compatible with my experience. It is certainly not true that a Linux box with 2000 threads blocked on i/o will be having any sort of bad time. If you're really got 2000 threads competing for CPU time then your problem transcends execution architecture: you've simply admitted more work than you can reasonably discharge. Neither threads nor callbacks can solve this problem for you. Also I'm not sure why you think that the kernel thread scheduler is all that good, or why it shouldn't be considered "outside our control." For my users the kernel thread scheduler is just another black object inside a dark box. The standard scheduler is pretty good for general purposes but I doubt its optimal for any particular case. In some loads it might be useful to yield to a specific thread that you think is holding a mutex your thread needs to acquire. This is cooperative multitasking, basically.
- plq 12y ago
  > It is certainly not true that a Linux box with 2000 threads blocked on i/o will be having any sort of bad time. You and I must have very different perceptions about the way a server is "having a bad time" :) I was assuming they were at various stages of processing an incoming request, which means they were blocked on either legitimate disk i/o or swapping. It's very difficult to log in even locally in that case, because the login process can't read /etc/passwd in time. > Also I'm not sure why you think that the kernel thread scheduler is all that good, or why it shouldn't be considered "outside our control. I was just trying to say that it's dangerous to give non-trusted peers big influence on the way the scheduler behaves.
  - thrownaway2424 12y ago
    OK, but it sounds like the main problem there is local disk access, which is the great satan anyway. You an generate a machine-hosing writeout workload using only one thread on Linux, because Linux loves to starve readers if it can write instead. I think the unstated second dimension of your comment is that most operating system distributions come out of the installer with absolutely the wrong parameters for running many threads. I think the default thread stack size is 2MB still, and the socket buffers are all huge, and there are limits on how many processes you can have that make people think those limits are meaningful, when they're really not.
rdtsc 12y ago
> Cooperative multitasking doesn't magically gzip your instruction pipeline or something (actually CM is suboptimal). There are multiple meanings of speed in a concurrent system. Is it throughput of each individual connection? Average throughput? Is it minimul latency, is it average latency? What cooperative multi-tasking does is let all tasks make progress. Without having to have an explicit scheduling algorithm in each individual task. There could be many reasons why ssh-ing migth be slow in a one-thread per connection. It could be that memory is low. Maybe the machine is over capacity. Maybe it is good that your ssh is slow because some higher business priority data is being transferred. Thread also doesn't necessarily mean OS threads. Threads could be mean green threads (as in Python greenlet co-routines), it could be Go's channels, it could be Erlang's processes. Those are N:M threading models. For example WhatsApp was running with 2M concurrent TCP connection on their FreeBSD servers say 3-4 years ago. Each connection with a separate lightweight process. So the logical model works well. It is the platforms and langauges that are behind so to speak.

by plq 12y ago

thrownaway2424 12y ago

"Ever tried to ssh into a one-thread-per-connection setup under heavy load? Assuming you managed to log in, it'll be very very difficult to get htop to execute when it's competing with 2k other processes for cpu time." Nothing about your comment is compatible with my experience. It is certainly not true that a Linux box with 2000 threads blocked on i/o will be having any sort of bad time. If you're really got 2000 threads competing for CPU time then your problem transcends execution architecture: you've simply admitted more work than you can reasonably discharge. Neither threads nor callbacks can solve this problem for you. Also I'm not sure why you think that the kernel thread scheduler is all that good, or why it shouldn't be considered "outside our control." For my users the kernel thread scheduler is just another black object inside a dark box. The standard scheduler is pretty good for general purposes but I doubt its optimal for any particular case. In some loads it might be useful to yield to a specific thread that you think is holding a mutex your thread needs to acquire. This is cooperative multitasking, basically.

plq 12y ago
> It is certainly not true that a Linux box with 2000 threads blocked on i/o will be having any sort of bad time. You and I must have very different perceptions about the way a server is "having a bad time" :) I was assuming they were at various stages of processing an incoming request, which means they were blocked on either legitimate disk i/o or swapping. It's very difficult to log in even locally in that case, because the login process can't read /etc/passwd in time. > Also I'm not sure why you think that the kernel thread scheduler is all that good, or why it shouldn't be considered "outside our control. I was just trying to say that it's dangerous to give non-trusted peers big influence on the way the scheduler behaves.
- thrownaway2424 12y ago
  OK, but it sounds like the main problem there is local disk access, which is the great satan anyway. You an generate a machine-hosing writeout workload using only one thread on Linux, because Linux loves to starve readers if it can write instead. I think the unstated second dimension of your comment is that most operating system distributions come out of the installer with absolutely the wrong parameters for running many threads. I think the default thread stack size is 2MB still, and the socket buffers are all huge, and there are limits on how many processes you can have that make people think those limits are meaningful, when they're really not.

rdtsc 12y ago

> Cooperative multitasking doesn't magically gzip your instruction pipeline or something (actually CM is suboptimal). There are multiple meanings of speed in a concurrent system. Is it throughput of each individual connection? Average throughput? Is it minimul latency, is it average latency? What cooperative multi-tasking does is let all tasks make progress. Without having to have an explicit scheduling algorithm in each individual task. There could be many reasons why ssh-ing migth be slow in a one-thread per connection. It could be that memory is low. Maybe the machine is over capacity. Maybe it is good that your ssh is slow because some higher business priority data is being transferred. Thread also doesn't necessarily mean OS threads. Threads could be mean green threads (as in Python greenlet co-routines), it could be Go's channels, it could be Erlang's processes. Those are N:M threading models. For example WhatsApp was running with 2M concurrent TCP connection on their FreeBSD servers say 3-4 years ago. Each connection with a separate lightweight process. So the logical model works well. It is the platforms and langauges that are behind so to speak.