3 ms·

In order for something to be "bound" by something else there must be an element of concurrency. People often forget this and I think it confuses the definition.

In order for something to be "bound" by something else there must be an element of concurrency. People often forget this and I think it confuses the definition. At this point people say "bound by X" when what they really mean is "most of the time is spent doing X". If you spend 5 seconds doing I/O and then afterwards 1 second doing CPU heavy work, you are not I/O bound. You can tell this because making the CPU work take 0.5 seconds less time still saves you 0.5 seconds. It's not a bound at all. In order for this to be I/O bound you would need to do the CPU heavy work during the I/O. Once the task is done that way, any optimisation on the CPU heavy work has precisely zero effect on runtime. This is how you know it's bound. I think the reason that web programmers have become confused by this particularly is because on a web server there is almost always inter-request concurrency. You run a bunch of workers on the same CPU and now your waiting-for-db time is concurrent with the doing-cpu-work time for a different request. The system as a whole can be considered I/O bound here because no mater how much faster you make the CPU, the number of requests per second you can process doesn't go up and so it seems like you might as well not care about how fast the cpu-work part is. And that's all well and good, but it's cold comfort for any individual request that still has to wait the full amount of time for a request to be served. You could absolutely improve the situation for any given request by improving the speed of the CPU or optimising the code it runs.

HeavenFox 6y ago
While correct, I think that's a bit too pedantic. For example, if you have a program that spend 5 seconds doing I/O and 0.05 second in CPU, for all intents and purposes it's I/O bound. Given the fast CPUs in today's world, this kind of workload is pretty common.
- wtallis 6y ago
  It's probably better to look at things proportionally, rather than in terms of absolute time saved. And it's definitely more useful to consider terms like "IO bound" and "CPU bound" as relative and approximate, rather than unattainable absolutes. Using the original numbers of 5 seconds IO, 1 second CPU: If doubling the speed of the CPU portion only makes the overall process 8% faster, it's not meaningfully CPU bound. But doubling the IO speed would reduce the total job time by 41%, which is a pretty big speedup overall. Using your example of 5 seconds IO and 0.05 seconds CPU: doubling the CPU speed gives a 0.5% overall reduction in job time, and doubling the IO performance gives a 49.5% overall reduction. So this example can be meaningfully said to be more IO bound than the first example.
- doonesbury 6y ago
  Well, almost. A good pedantic definition would show how to see i/o rates as a percentage of maximum possible. And if those numbers were quite high and the CPU part was in the i/o loop (r/w, then compute, repeat) then indeed i/o bound. No amount of work on the CPU part will appreciably drive down absolute time. If the CPU part is only after i/o the CPU improvements can at most only help relatively little but at least CPU delays isn't making i/o artificially slower by delaying when new i/o can start. What's a little misleading in the example is that the problem setup already is a half answer. In complex commercial apps just arriving at an apportionment of 5 and .05 for some narrow set of use cases is already an achievement. If we see 5 and .05 comes in the mixed case (CPU in the i/o loop) then there is a choice on what to fix to drive down absolute time. If the numbers are not so skewed it may take more debugging to suss out what's going on. Here it is obvious with givens. If apart we conclude the i/o is slow (pedantically i/o bound) and there's no engineering need to figure out where labor needs to go. Here it's pointless to wonder about CPU or memory. The emphasis in sum is on the management end: our system slow. But what are we going to fix and why.

HeavenFox 6y ago

While correct, I think that's a bit too pedantic. For example, if you have a program that spend 5 seconds doing I/O and 0.05 second in CPU, for all intents and purposes it's I/O bound. Given the fast CPUs in today's world, this kind of workload is pretty common.

wtallis 6y ago
It's probably better to look at things proportionally, rather than in terms of absolute time saved. And it's definitely more useful to consider terms like "IO bound" and "CPU bound" as relative and approximate, rather than unattainable absolutes. Using the original numbers of 5 seconds IO, 1 second CPU: If doubling the speed of the CPU portion only makes the overall process 8% faster, it's not meaningfully CPU bound. But doubling the IO speed would reduce the total job time by 41%, which is a pretty big speedup overall. Using your example of 5 seconds IO and 0.05 seconds CPU: doubling the CPU speed gives a 0.5% overall reduction in job time, and doubling the IO performance gives a 49.5% overall reduction. So this example can be meaningfully said to be more IO bound than the first example.
doonesbury 6y ago
Well, almost. A good pedantic definition would show how to see i/o rates as a percentage of maximum possible. And if those numbers were quite high and the CPU part was in the i/o loop (r/w, then compute, repeat) then indeed i/o bound. No amount of work on the CPU part will appreciably drive down absolute time. If the CPU part is only after i/o the CPU improvements can at most only help relatively little but at least CPU delays isn't making i/o artificially slower by delaying when new i/o can start. What's a little misleading in the example is that the problem setup already is a half answer. In complex commercial apps just arriving at an apportionment of 5 and .05 for some narrow set of use cases is already an achievement. If we see 5 and .05 comes in the mixed case (CPU in the i/o loop) then there is a choice on what to fix to drive down absolute time. If the numbers are not so skewed it may take more debugging to suss out what's going on. Here it is obvious with givens. If apart we conclude the i/o is slow (pedantically i/o bound) and there's no engineering need to figure out where labor needs to go. Here it's pointless to wonder about CPU or memory. The emphasis in sum is on the management end: our system slow. But what are we going to fix and why.