4 ms·

I'd be interested to understand why you would classify web servers as generally I/O bound, rather than CPU (or memory) bound.

by nbm 14y ago

nivertech 14y ago
What web application server will usually do is to accept HTTP request via TCP, fetch/update data in database server, render a web page and send HTTP response back via TCP. So essentially it just waiting, either for TCP stack or for database driver. IO-bound. Facebook photos application for example, can be CPU-bound, because image processing applications had to process millions of pixels efficiently.
- nbm 14y ago
  Why can't the web server do other work with the CPU and other resources at the same time that it is waiting for those network operations to happen?
  - nivertech 14y ago
    Yes, but what the other threads will do? Answer: same stuff. I can put it another way: Some Tilera boards have 4x10GbE ports, but have no good floating point support. On the other hand Xeon Phi has AVX2 (512 bit wide SIMD) with excellent floating point performance, but TCP/IP stack emulated over PCI-express. So which one will you choose for network server and which for number crunching?
    - nbm 14y ago
      My somewhat circuitous point is that the ability of a server to perform work is bound by something that runs out - CPU (no more CPU time available), disk I/O (can't handle more I/O requests to disk), network I/O (can't put or get any more packets through your network card or some upstream device/pipe), and so forth. Colloquially, whatever runs out first is what you are bound by. You can either increase the resource (upgrading CPU, adding memory), or use the resource in a different way/trade off one resource against another. For example, compressing data before network or disk I/O might be useful if you have spare CPU. Once you've done that, you may now be bound by something else - maybe available memory. If you have spare resources (CPU, for example), you can then choose to use fewer/less powerful resources in that space if it makes economic sense. You can also talk about response time being bound by something - disk seek time, network latency, memory latency, and so forth. That affects how fast something can respond, and while that figures into how much work a server can do (per-thread/request memory usage), it doesn't generally largely affect how many machines you have to buy or their configuration (unless you have low-latency options).
- aristus 14y ago
  (FB employee, tangentially related to this stuff) Everyone, including myself, casually uses the term "CPU-bound", "I/O bound", "Memory-bound" for convenience but the truth is usually more complex. You might actually be CPU-instruction-cache-bound or memory-latency-bound. One app I worked on a long time ago turned out to be random-numbers-bound. :) In this context boundedness means roughly "the thing that would increase throughput if we had relatively more of it". A pizza shop is going to be fundamentally "oven-bound", because everything else from employees to counter space can be increased more easily than the size of the oven. Waiting on outside I/O (databases) does not mean that a webserver is I/O bound. The system as a whole might be I/O bound, but not the web layer. Threads stuck waiting for dataz are just put to sleep and others take over the CPU.
  - voltagex_ 14y ago
    Were you running out of entropy? If so, how did you solve it? (If you're allowed to divulge that)
    - aristus 14y ago
      With a Heisenberg Compensator! No. Like with any bottlenecked resource, you first measure how much you are using per "transaction", whatever that may be. Then you see whether you can use what you have more efficiently. Maybe you don't need it at all. Or maybe you can cut corners by using /dev/urandom or something. But suppose you are generating tons of SSL certificates. You can't --or shouldn't-- cut corners. For example, see https://freedom-to-tinker.com/blog/nadiah/new-research-theres-no-need-panic-over-factorable-keys-just-mind-your-ps-and-qs/ https://freedom-to-tinker.com/blog/nadiah/new-research-there... So... what do you do? The problem, like almost any bottleneck problem, isn't that you don't have "enough" of the resource, it's that the resource is out of balance with the other resources like CPU or RAM. When you think about it that way a lot of possibilities open up, like installing special cards or dongles that generate entropy, or by buying lots of lower-powered devices that have more entropy relative to their CPU throughput.

I'd be interested to understand why you would classify web servers as generally I/O bound, rather than CPU (or memory) bound.

by nbm 14y ago

I'd be interested to understand why you would classify web servers as generally I/O bound, rather than CPU (or memory) bound.

nivertech 14y ago

What web application server will usually do is to accept HTTP request via TCP, fetch/update data in database server, render a web page and send HTTP response back via TCP. So essentially it just waiting, either for TCP stack or for database driver. IO-bound. Facebook photos application for example, can be CPU-bound, because image processing applications had to process millions of pixels efficiently.

nbm 14y ago
Why can't the web server do other work with the CPU and other resources at the same time that it is waiting for those network operations to happen?
- nivertech 14y ago
  Yes, but what the other threads will do? Answer: same stuff. I can put it another way: Some Tilera boards have 4x10GbE ports, but have no good floating point support. On the other hand Xeon Phi has AVX2 (512 bit wide SIMD) with excellent floating point performance, but TCP/IP stack emulated over PCI-express. So which one will you choose for network server and which for number crunching?
  - nbm 14y ago
    My somewhat circuitous point is that the ability of a server to perform work is bound by something that runs out - CPU (no more CPU time available), disk I/O (can't handle more I/O requests to disk), network I/O (can't put or get any more packets through your network card or some upstream device/pipe), and so forth. Colloquially, whatever runs out first is what you are bound by. You can either increase the resource (upgrading CPU, adding memory), or use the resource in a different way/trade off one resource against another. For example, compressing data before network or disk I/O might be useful if you have spare CPU. Once you've done that, you may now be bound by something else - maybe available memory. If you have spare resources (CPU, for example), you can then choose to use fewer/less powerful resources in that space if it makes economic sense. You can also talk about response time being bound by something - disk seek time, network latency, memory latency, and so forth. That affects how fast something can respond, and while that figures into how much work a server can do (per-thread/request memory usage), it doesn't generally largely affect how many machines you have to buy or their configuration (unless you have low-latency options).
aristus 14y ago
(FB employee, tangentially related to this stuff) Everyone, including myself, casually uses the term "CPU-bound", "I/O bound", "Memory-bound" for convenience but the truth is usually more complex. You might actually be CPU-instruction-cache-bound or memory-latency-bound. One app I worked on a long time ago turned out to be random-numbers-bound. :) In this context boundedness means roughly "the thing that would increase throughput if we had relatively more of it". A pizza shop is going to be fundamentally "oven-bound", because everything else from employees to counter space can be increased more easily than the size of the oven. Waiting on outside I/O (databases) does not mean that a webserver is I/O bound. The system as a whole might be I/O bound, but not the web layer. Threads stuck waiting for dataz are just put to sleep and others take over the CPU.
- voltagex_ 14y ago
  Were you running out of entropy? If so, how did you solve it? (If you're allowed to divulge that)
  - aristus 14y ago
    With a Heisenberg Compensator! No. Like with any bottlenecked resource, you first measure how much you are using per "transaction", whatever that may be. Then you see whether you can use what you have more efficiently. Maybe you don't need it at all. Or maybe you can cut corners by using /dev/urandom or something. But suppose you are generating tons of SSL certificates. You can't --or shouldn't-- cut corners. For example, see https://freedom-to-tinker.com/blog/nadiah/new-research-theres-no-need-panic-over-factorable-keys-just-mind-your-ps-and-qs/ https://freedom-to-tinker.com/blog/nadiah/new-research-there... So... what do you do? The problem, like almost any bottleneck problem, isn't that you don't have "enough" of the resource, it's that the resource is out of balance with the other resources like CPU or RAM. When you think about it that way a lot of possibilities open up, like installing special cards or dongles that generate entropy, or by buying lots of lower-powered devices that have more entropy relative to their CPU throughput.