3 ms·
io_uring is no more specialized than an SSD is today, our storage interfaces went from handling between 1 and 4 requests in parallel for the past 50 years to su
by dmw_ng 6y ago
io_uring is no more specialized than an SSD is today, our storage interfaces went from handling between 1 and 4 requests in parallel for the past 50 years to suddenly handling literally hundreds on consumer grade hardware (IIRC NVMe is specced either for up to 64k queue depth or unlimited queue depth). Of course the software environment must change to keep up, is it really reasonable to continue feeding such devices one IO at a time because that's how it was done in 1970?
It is not fun to program against io_uring just as BPF can be a nightmare, but that's a problem for userspace to solve with libraries and better abstractions, much in the same way userspace usually doesn't have to deal with parsing the shared ELF segment exported by the kernel (glibc does that) or writing complex routines for fetching the system time (the ELF segment does that), both of which are implementation details necessary for extracting the full performance from modern hardware.
We'll catch up eventually, but first the lower level interfaces must exist. In meantime, I cringe Every. Single. Time. I run UNIX find or du over my home directory, realizing it could have completed in a fraction of the time for almost 10 years now if only our traditional software environment was awakened to the reality of the hardware it has long since run on.
- erk__ 6y agoWell at least in the near future I don't think you will see it on any other UNIX than Linux as it is compared to aio not a standard, but a Linux specific api.
- dmw_ng 6y agoBPF and io_uring architectural styles both have heritage in and close equivalents at least on BSD. You could consider io_uring as nothing but a slightly fancier cousin of FreeBSD's netmap that exists for exactly the same reason, they just wear different shades of lipstick. BPF of course came directly from BSD into Linux Any inevitable standard will likely come from userspace, much in the same way libpcap successfully papered over raw packet capture for a huge variety of operating systems. The underlying kernel interface is basically just details. We're decades past the point where application portability required standards like POSIX to make progress, I imagine comparatively few modern programmers even know much about those kinds of standards any more.
- floatboth 6y agoHuh? io_uring from what I've seen is mostly about direct disk async IO which some databases like (and I prefer databases that mmap :P) while netmap is kernel bypass for networking – "mmap" for NIC ring buffers – without monster frameworks like DPDK, with universal API that keeps all NIC-speicifics in the kernel drivers.
- dmw_ng 6y agoCousins not twins.. io_uring with O_DIRECT to a block device is already architecturally equivalent to netmap with a network device. Its kernel-side internals have been kept sufficiently generic that we might eventually even see functional equivalence (but if Axboe is listening, please give us getdents64() via uring first!).
- erk__ 6y agoHopefully you are right and software will not not being to lean too much on direct io_uring instead of some kind of wrapper. Thanks for the comment it was pretty educational.