4 ms·

A context switch takes about 50 microseconds. If you want to handle 10,000 requests in a second 500 milliseconds is lost to context switching alone. Probably mo

by dclusin 11y ago

A context switch takes about 50 microseconds. If you want to handle 10,000 requests in a second 500 milliseconds is lost to context switching alone. Probably more. The nature of IO & user input in general is to hurry up and wait. It's usually simpler to do async requests & polling file descriptors rather than thread-per-request. The former seems on the surface more complicated and more lines of code, but IMO, ends up being less complicated because you won't have to spend time digging through logs trying to figure out that the reason some other unrelated thread stalled for so long is because you were blasted with thousands of requests in the same second.

thrownaway2424 11y ago
Where did you get that number? That's the kind of SunOS derangement I'm talking about. On Linux with NPTL (which is over a decade old) and any recent Intel CPU (Westmere or later) a context switch between threads of the same process takes only about 1-2 microseconds.
- jcoffland 11y ago
  Regardless, context switching, waiting on locks and the added cost of allocating a stack to each thread are all relatively expensive. I've gotten huge performance increases by rewriting threaded code to instead use non-blocking IO. Nginx vs. Apache is a great example of this difference. https://www.nginx.com/blog/inside-nginx-how-we-designed-for-performance-scale/ https://www.nginx.com/blog/inside-nginx-how-we-designed-for-...
  - thrownaway2424 11y ago
    I think these can be and often are combined. You can definitely have an event driven main IO loop accepting connections and reading packets, that nevertheless dispatches whole requests onto their own threads. I work with a lot of services written that way. The fact that you can reuse the frontend for several applications amortizes the cost of developing it as a high performance asynchronous thing. As for stack allocation I definitely see your point there. It is bizarre that Linux defaults to 2MB or whatever it is these days. I usually work with a 64KB thread stack.
jcoffland 11y ago
This is exactly what I'm talking about. If you want to reach really high transactions rates then you cannot use one thread per transaction.
- thrownaway2424 11y ago
  I won't argue that async software thread per hardware thread style will have better maximum performance than a thread-per-request style, but I think that difference will not be material in most applications (the difference between the two styles of C++ server programming being far smaller than the difference between, say, a Go program and a C++ program), and the cost paid in code readability is pretty high. All that async continuation passing makes your code a twisty maze, and that style is fraught with lifetime problems that are the very thing most C++ haters are complaining about. In synchronous style all the allocations are scoped by nature. In async style the programmer has to assign ownership of everything to some master continuation that will delete them all in the final callback. That adds a lot of code.
  - jcoffland 11y ago
    You are right about the added difficulty of resource ownership in the face of asynchronous code. This is a trade off. lambda functions in C++11 can help with this. They make it possible to pass resource allocations (smart pointers) through to callbacks.
  - zzzcpan 11y ago
    > All that async continuation passing makes your code a twisty maze On the contrary, it gives your code clarity, consistency and predictability. > some master continuation that will delete them all in the final callback Even nginx in plain C is not doing that, but simply creates destructors on allocation to avoid tracking stuff manually. Just don't jump into the first thing you read somewhere, remember one simple rule: if it feels harder than threads - you are doing it wrong.
    - SamReidHughes 11y ago

by dclusin 11y ago

thrownaway2424 11y ago

Where did you get that number? That's the kind of SunOS derangement I'm talking about. On Linux with NPTL (which is over a decade old) and any recent Intel CPU (Westmere or later) a context switch between threads of the same process takes only about 1-2 microseconds.

jcoffland 11y ago
Regardless, context switching, waiting on locks and the added cost of allocating a stack to each thread are all relatively expensive. I've gotten huge performance increases by rewriting threaded code to instead use non-blocking IO. Nginx vs. Apache is a great example of this difference. https://www.nginx.com/blog/inside-nginx-how-we-designed-for-performance-scale/ https://www.nginx.com/blog/inside-nginx-how-we-designed-for-...
- thrownaway2424 11y ago
  I think these can be and often are combined. You can definitely have an event driven main IO loop accepting connections and reading packets, that nevertheless dispatches whole requests onto their own threads. I work with a lot of services written that way. The fact that you can reuse the frontend for several applications amortizes the cost of developing it as a high performance asynchronous thing. As for stack allocation I definitely see your point there. It is bizarre that Linux defaults to 2MB or whatever it is these days. I usually work with a 64KB thread stack.

jcoffland 11y ago

This is exactly what I'm talking about. If you want to reach really high transactions rates then you cannot use one thread per transaction.

thrownaway2424 11y ago
I won't argue that async software thread per hardware thread style will have better maximum performance than a thread-per-request style, but I think that difference will not be material in most applications (the difference between the two styles of C++ server programming being far smaller than the difference between, say, a Go program and a C++ program), and the cost paid in code readability is pretty high. All that async continuation passing makes your code a twisty maze, and that style is fraught with lifetime problems that are the very thing most C++ haters are complaining about. In synchronous style all the allocations are scoped by nature. In async style the programmer has to assign ownership of everything to some master continuation that will delete them all in the final callback. That adds a lot of code.
- jcoffland 11y ago
  You are right about the added difficulty of resource ownership in the face of asynchronous code. This is a trade off. lambda functions in C++11 can help with this. They make it possible to pass resource allocations (smart pointers) through to callbacks.
- zzzcpan 11y ago
  > All that async continuation passing makes your code a twisty maze On the contrary, it gives your code clarity, consistency and predictability. > some master continuation that will delete them all in the final callback Even nginx in plain C is not doing that, but simply creates destructors on allocation to avoid tracking stuff manually. Just don't jump into the first thing you read somewhere, remember one simple rule: if it feels harder than threads - you are doing it wrong.
  - SamReidHughes 11y ago