4 ms·

Hey Static, thanks for the thought-provoking dicussion the other day! Let me ask you this: let's say you're downstream of a very slow server that is outside of

by btrask 6y ago

Hey Static, thanks for the thought-provoking dicussion the other day! Let me ask you this: let's say you're downstream of a very slow server that is outside of your control. You need to access it, and there's nothing you can do to speed it up. Are you IO bound in this case?

staticassertion 6y ago
Honestly, this is a reasonable question. I'm not really sure, and I think this is a gap in terminology (or even tooling), as I think we both sort of stumbled into the other day. I want to say no? If I were to reframe the question as a slow function that I had to call, I wouldn't call it IO bound, so I think not - the fact that there's IO in between the two components is more of a confounding factor in profiling than an actual performance limitation.
- btrask 6y ago
  Good answer :)
- kbenson 6y ago
  I've definitely seen it referred to as IO bound when an upstream server just can't get your input fast enough, whether that's because you've maxed out the medium (e.g. you're bound by a 10/100/1000 Mbit link), or bound by how fast the disks can serve up bits, to downstream it's generally encountered as the same thing, the ability to process IO faster than it's available.
GauntletWizard 6y ago
In this case, I call it “dependency bound”. Anything not in your direct control is a dependency, and if that’s your limiting factor, call it such.
hinkley 6y ago
If you're on a group hike/bike/ride/run, the group goes at the speed of the slowest person. It may be that you didn't sleep well last night, or that your stomach is bugging you, but the reason it's going to take an extra hour today is because Tim has a blister. Whatever is holding up that server is what's holding up the entire train of communication. If that is where $5000 will fix the problem, don't talk about any other problems, you'll just confuse management. The next place to stop if that doesn't work is, if you can't fix "Tim's foot" can you offload some things "he" is dealing with? Dump some of his load, work steal, etc. But these are just mitigations.
- Jtsummers 6y ago
  Somebody read The Goal. But this is the summary of a large part of that book: Systems have bottlenecks, address the bottleneck. Spending time on improvements elsewhere will not help the bottleneck, you're just wasting time/money. Data/material is going into it too fast to process (work is building up). Improving before the bottleneck just makes its queue fill faster. Improving after the bottleneck just creates a segment of the system that's starved for work. So focus on the problem at hand, once it's addressed, focus on the next bottleneck.
  - hinkley 6y ago
    That someone was a coworker, but a bit of this is just queuing theory. Buffering up does improve things when there is variability the processing time for each task, but it can also make average wait time hell. So you better be sure whether latency or throughput is really your primary concern.
Volundr 6y ago
I'm not sure I'd call this "bound" at all, though I like the siblings "dependency bound". To me this would be "blocked". "Bound" means I'm doing something productive, and I'm "bound" on whatever is limiting my speed. If I'm waiting on an external API outside my control, I'm not "bound" I'm "blocked" (stuck sitting around doing nothing, making no progress until they deign to get back to me).
- coldtea 6y ago
  Isn't a "bottleneck" the traditional term for this?
- hevelvarik 6y ago
  This is a nice and more technically accurate definition but i don’t believe it’s at all found in the wild, which mitigates its usefulness. I think the bound in the common usage is in the sense of destination rather than limit, ie, bound for an i/o subsystem whether under local control or not. This then implies the local actor entering the ‘blocked’ state that you mention, during which time the local actor is free in principle to do something else. In short I think I/O bound is a concise way of saying that this work will leave my concern and will pass back into it at some later time.
  - Jtsummers 6y ago
    IO and CPU bound are meant in the sense of upper and lower bound, not destination, but limits on the system. Saying something is CPU bound means that the CPU is the bottleneck. For instance, the data IO doesn’t limit the system throughout of iteratively running a hash algorithm a million times, the CPU does. An IO bound process might be something like reading a lot of data off a disk (or over a network) with a small, quick transformation on the CPU. Where you saturate the data bus, but the CPU is comparatively idle.
    - hevelvarik 6y ago
      Ok hmm, thank you for the correction.

by btrask 6y ago

staticassertion 6y ago

Honestly, this is a reasonable question. I'm not really sure, and I think this is a gap in terminology (or even tooling), as I think we both sort of stumbled into the other day. I want to say no? If I were to reframe the question as a slow function that I had to call, I wouldn't call it IO bound, so I think not - the fact that there's IO in between the two components is more of a confounding factor in profiling than an actual performance limitation.

btrask 6y ago
Good answer :)
kbenson 6y ago
I've definitely seen it referred to as IO bound when an upstream server just can't get your input fast enough, whether that's because you've maxed out the medium (e.g. you're bound by a 10/100/1000 Mbit link), or bound by how fast the disks can serve up bits, to downstream it's generally encountered as the same thing, the ability to process IO faster than it's available.

GauntletWizard 6y ago

In this case, I call it “dependency bound”. Anything not in your direct control is a dependency, and if that’s your limiting factor, call it such.

hinkley 6y ago

If you're on a group hike/bike/ride/run, the group goes at the speed of the slowest person. It may be that you didn't sleep well last night, or that your stomach is bugging you, but the reason it's going to take an extra hour today is because Tim has a blister. Whatever is holding up that server is what's holding up the entire train of communication. If that is where $5000 will fix the problem, don't talk about any other problems, you'll just confuse management. The next place to stop if that doesn't work is, if you can't fix "Tim's foot" can you offload some things "he" is dealing with? Dump some of his load, work steal, etc. But these are just mitigations.

Jtsummers 6y ago
Somebody read The Goal. But this is the summary of a large part of that book: Systems have bottlenecks, address the bottleneck. Spending time on improvements elsewhere will not help the bottleneck, you're just wasting time/money. Data/material is going into it too fast to process (work is building up). Improving before the bottleneck just makes its queue fill faster. Improving after the bottleneck just creates a segment of the system that's starved for work. So focus on the problem at hand, once it's addressed, focus on the next bottleneck.
- hinkley 6y ago
  That someone was a coworker, but a bit of this is just queuing theory. Buffering up does improve things when there is variability the processing time for each task, but it can also make average wait time hell. So you better be sure whether latency or throughput is really your primary concern.

Volundr 6y ago

I'm not sure I'd call this "bound" at all, though I like the siblings "dependency bound". To me this would be "blocked". "Bound" means I'm doing something productive, and I'm "bound" on whatever is limiting my speed. If I'm waiting on an external API outside my control, I'm not "bound" I'm "blocked" (stuck sitting around doing nothing, making no progress until they deign to get back to me).

coldtea 6y ago
Isn't a "bottleneck" the traditional term for this?
hevelvarik 6y ago
This is a nice and more technically accurate definition but i don’t believe it’s at all found in the wild, which mitigates its usefulness. I think the bound in the common usage is in the sense of destination rather than limit, ie, bound for an i/o subsystem whether under local control or not. This then implies the local actor entering the ‘blocked’ state that you mention, during which time the local actor is free in principle to do something else. In short I think I/O bound is a concise way of saying that this work will leave my concern and will pass back into it at some later time.
- Jtsummers 6y ago
  IO and CPU bound are meant in the sense of upper and lower bound, not destination, but limits on the system. Saying something is CPU bound means that the CPU is the bottleneck. For instance, the data IO doesn’t limit the system throughout of iteratively running a hash algorithm a million times, the CPU does. An IO bound process might be something like reading a lot of data off a disk (or over a network) with a small, quick transformation on the CPU. Where you saturate the data bus, but the CPU is comparatively idle.
  - hevelvarik 6y ago
    Ok hmm, thank you for the correction.