5 ms·

"Are you actually IO bound? I seriously doubt it." https://twitter.com/rygorous/status/1423778960739950595 https://twitter.com/rygorous/status/1423778960739950

"Are you actually IO bound? I seriously doubt it." https://twitter.com/rygorous/status/1423778960739950595 https://twitter.com/rygorous/status/1423778960739950595

kadoban 4y ago
That seems specific to games. Business logic is quite a bit more likely to just be waiting on IO, usually fetching from the DB or sending/receiving over the network. This is because there's a lot of business logic where those things are _everything it does_.
- hresvelgr 4y ago
  I think the point of the twitter rant was addressing a sentiment of "Oh, it's IO bound, guess I can't do anything about it." When there is usually something that can be done about it.
- janwas 4y ago
  Business logic with tons of branches is indeed not amenable to SIMD, nor is that where the compute is. In a world with 400 Gb/s NICs, being IO bound takes some doing. How much SW manages that without SIMD, even on multiple cores?
  - sakras 4y ago
    Until adoption of io_uring picks up (looking at you RHEL), I’m not sure it’s even really possible to hit the bandwidth of modern nvme drives. It takes a lot of parallelism to do, and eventually context switch overhead kills you.
    - janwas 4y ago
      FWIW Windows has had something basically equivalent (IO completion ports) for quite some time. Indeed helpful, I remember that early SSDs already required high queue depths.
  - Someone 4y ago
    > In a world with 400 Gb/s NICs, being IO bound takes some doing. depends on what you call I/O. From the perspective of the caller, that’s the entire stack starting at a http library call or sql driver call. If those can’t use the hardware to their fullest (or can’t, the way you’re using them) your process becomes I/O bound from its perspective.
    - janwas 4y ago
      Fair point :) I'm assuming the stack is doing what it can to avoid being the bottleneck. On a system level, I'd still call what you're describing CPU-bound, as in: unable to use all potential IO capacity.
  - gpderetta 4y ago
    > In a world with 400 Gb/s NICs, being IO bound takes some doing. Amen. CPU have stopped improving significantly a decade ago or more. IO never stopped.
  - kadoban 4y ago
    It doesn't matter if you're 400 Tb/s, you still can't help the speed of light. You're still going to be waiting for a round-trip or a few.
    - janwas 4y ago
      OK, if latency is a concern, is it possible to parallelize requests or increase batch sizes to maximize utilization and throughput?
captainmuon 4y ago
IO bound doesn't neccessarily mean waiting for GBs to transfer from the disk. It can mean reading meeelions of tiny files and seeking back and forth randomly. Doesn't matter if the files are in the cache or not. When someone says IO bound, it often means CPU bound but in the kernel or in some low level code. Most apps and games could open instantly, unless you are doing something like heavy map generation or hitting the network. But it would require structuring the app specifically for it and there is often no incentive to do so.
- the8472 4y ago
  > It can mean reading meeelions of tiny files and seeking back and forth randomly. Tiny files are fine if you fetch them from NVMe and keep the queues fed, i.e. you need to issue multiple concurrent reads, not do it sequentially. Not being on windows helps too.
pca006132 4y ago
Perhaps something can be memory bound instead of CPU or IO bound?

"Are you actually IO bound? I seriously doubt it." https://twitter.com/rygorous/status/1423778960739950595 https://twitter.com/rygorous/status/1423778960739950595

kadoban 4y ago

That seems specific to games. Business logic is quite a bit more likely to just be waiting on IO, usually fetching from the DB or sending/receiving over the network. This is because there's a lot of business logic where those things are _everything it does_.

hresvelgr 4y ago
I think the point of the twitter rant was addressing a sentiment of "Oh, it's IO bound, guess I can't do anything about it." When there is usually something that can be done about it.
janwas 4y ago
Business logic with tons of branches is indeed not amenable to SIMD, nor is that where the compute is. In a world with 400 Gb/s NICs, being IO bound takes some doing. How much SW manages that without SIMD, even on multiple cores?
- sakras 4y ago
  Until adoption of io_uring picks up (looking at you RHEL), I’m not sure it’s even really possible to hit the bandwidth of modern nvme drives. It takes a lot of parallelism to do, and eventually context switch overhead kills you.
  - janwas 4y ago
    FWIW Windows has had something basically equivalent (IO completion ports) for quite some time. Indeed helpful, I remember that early SSDs already required high queue depths.
- Someone 4y ago
  > In a world with 400 Gb/s NICs, being IO bound takes some doing. depends on what you call I/O. From the perspective of the caller, that’s the entire stack starting at a http library call or sql driver call. If those can’t use the hardware to their fullest (or can’t, the way you’re using them) your process becomes I/O bound from its perspective.
  - janwas 4y ago
    Fair point :) I'm assuming the stack is doing what it can to avoid being the bottleneck. On a system level, I'd still call what you're describing CPU-bound, as in: unable to use all potential IO capacity.
- gpderetta 4y ago
  > In a world with 400 Gb/s NICs, being IO bound takes some doing. Amen. CPU have stopped improving significantly a decade ago or more. IO never stopped.
- kadoban 4y ago
  It doesn't matter if you're 400 Tb/s, you still can't help the speed of light. You're still going to be waiting for a round-trip or a few.
  - janwas 4y ago
    OK, if latency is a concern, is it possible to parallelize requests or increase batch sizes to maximize utilization and throughput?

captainmuon 4y ago

IO bound doesn't neccessarily mean waiting for GBs to transfer from the disk. It can mean reading meeelions of tiny files and seeking back and forth randomly. Doesn't matter if the files are in the cache or not. When someone says IO bound, it often means CPU bound but in the kernel or in some low level code. Most apps and games could open instantly, unless you are doing something like heavy map generation or hitting the network. But it would require structuring the app specifically for it and there is often no incentive to do so.

the8472 4y ago
> It can mean reading meeelions of tiny files and seeking back and forth randomly. Tiny files are fine if you fetch them from NVMe and keep the queues fed, i.e. you need to issue multiple concurrent reads, not do it sequentially. Not being on windows helps too.

pca006132 4y ago

Perhaps something can be memory bound instead of CPU or IO bound?