3 ms·
Business logic with tons of branches is indeed not amenable to SIMD, nor is that where the compute is. In a world with 400 Gb/s NICs, being IO bound takes some
by janwas 4y ago
Business logic with tons of branches is indeed not amenable to SIMD, nor is that where the compute is.
In a world with 400 Gb/s NICs, being IO bound takes some doing. How much SW manages that without SIMD, even on multiple cores?
- sakras 4y agoUntil adoption of io_uring picks up (looking at you RHEL), I’m not sure it’s even really possible to hit the bandwidth of modern nvme drives. It takes a lot of parallelism to do, and eventually context switch overhead kills you.
- janwas 4y agoFWIW Windows has had something basically equivalent (IO completion ports) for quite some time. Indeed helpful, I remember that early SSDs already required high queue depths.
- Someone 4y ago> In a world with 400 Gb/s NICs, being IO bound takes some doing. depends on what you call I/O. From the perspective of the caller, that’s the entire stack starting at a http library call or sql driver call. If those can’t use the hardware to their fullest (or can’t, the way you’re using them) your process becomes I/O bound from its perspective.
- janwas 4y agoFair point :) I'm assuming the stack is doing what it can to avoid being the bottleneck. On a system level, I'd still call what you're describing CPU-bound, as in: unable to use all potential IO capacity.
- gpderetta 4y ago> In a world with 400 Gb/s NICs, being IO bound takes some doing. Amen. CPU have stopped improving significantly a decade ago or more. IO never stopped.
- kadoban 4y agoIt doesn't matter if you're 400 Tb/s, you still can't help the speed of light. You're still going to be waiting for a round-trip or a few.
- janwas 4y agoOK, if latency is a concern, is it possible to parallelize requests or increase batch sizes to maximize utilization and throughput?