3 ms·

No it will not. Two rather specialized tools to help with rather specific issues are no reason to throw out heaps and mounds of existing and perfectly working c

No it will not. Two rather specialized tools to help with rather specific issues are no reason to throw out heaps and mounds of existing and perfectly working code and solutions

kevincox 6y ago
There are tons of applications that are built upon higher-level APIs that can be updated to take advantage of this automatically. Furthermore you don't need to have every single application updated to revolutionize something. The apps that need this will be quite happy to make the jump.
harporoeder 6y ago
I expect these technologies to be integrated into language runtimes and webservers such that most developers won't even know it being used similar to EPOLL today. While revolutionary is an extreme characterization the performance and improved API changes are at least non trivial.
dclusin 6y ago
There's a very niche conference called eBPF Summit* that has presentations from people at companies with first class engineering orgs talking about what they are doing with eBPF. The problems they are solving and the breadth of problems being solved are very impressive. [*] - https://ebpf.io/summit-2020/ https://ebpf.io/summit-2020/
- jcelerier 6y ago
  yep, at the Linux Audio Conference we have a paper for using eBPF to process network audio packets this year for instance (https://lac2020.sciencesconf.org/program https://lac2020.sciencesconf.org/program / paper is https://lac2020.sciencesconf.org/307835 https://lac2020.sciencesconf.org/307835) - presentation is tomorrow (happens online)
- smartbit 6y ago
  This conference was more a Cilium conference. Google will use cilium as the default cni on gke https://news.ycombinator.com/item?id=24212021 https://news.ycombinator.com/item?id=24212021 Not speaking was Kinvolk’s CTO Alban Crequy https://news.ycombinator.com/item?id=23042722 https://news.ycombinator.com/item?id=23042722
dmw_ng 6y ago
io_uring is no more specialized than an SSD is today, our storage interfaces went from handling between 1 and 4 requests in parallel for the past 50 years to suddenly handling literally hundreds on consumer grade hardware (IIRC NVMe is specced either for up to 64k queue depth or unlimited queue depth). Of course the software environment must change to keep up, is it really reasonable to continue feeding such devices one IO at a time because that's how it was done in 1970? It is not fun to program against io_uring just as BPF can be a nightmare, but that's a problem for userspace to solve with libraries and better abstractions, much in the same way userspace usually doesn't have to deal with parsing the shared ELF segment exported by the kernel (glibc does that) or writing complex routines for fetching the system time (the ELF segment does that), both of which are implementation details necessary for extracting the full performance from modern hardware. We'll catch up eventually, but first the lower level interfaces must exist. In meantime, I cringe Every. Single. Time. I run UNIX find or du over my home directory, realizing it could have completed in a fraction of the time for almost 10 years now if only our traditional software environment was awakened to the reality of the hardware it has long since run on.
- erk__ 6y ago
  Well at least in the near future I don't think you will see it on any other UNIX than Linux as it is compared to aio not a standard, but a Linux specific api.
  - dmw_ng 6y ago
    BPF and io_uring architectural styles both have heritage in and close equivalents at least on BSD. You could consider io_uring as nothing but a slightly fancier cousin of FreeBSD's netmap that exists for exactly the same reason, they just wear different shades of lipstick. BPF of course came directly from BSD into Linux Any inevitable standard will likely come from userspace, much in the same way libpcap successfully papered over raw packet capture for a huge variety of operating systems. The underlying kernel interface is basically just details. We're decades past the point where application portability required standards like POSIX to make progress, I imagine comparatively few modern programmers even know much about those kinds of standards any more.
cpuguy83 6y ago
Neither ebpf or io_uring are "specialized", in fact they are specifically not specialized which is what makes them revolutionary (along with being better than currently specialized apis like aio).
gHosts 6y ago
What excites me is if you're working down at the high frequency, low latency I/O domain... linux traditionally has actually sort of sucked. We're _very_ good at buffering things up and handling large chunks to gain high throughput. We have layers and layers of magic that make us _very_ good at that. For cases where you have lots of devices throwing small chunks at high frequency and you have to respond (in this one, do a small computation, and out that one...) we've sucked bad. That's why bare metal RTOS's still exist. The io_uring / eBPF combo looks _very_ promising for opening up that domain in a tidy fashion. I also hope there will be no reason to throw away "heaps and mounds of existing and perfectly working code and solutions". I hope to replace my finely tuned epoll / read/write reactor pattern inner loop with io_uring / eBPF and leave 99% of the code untouched... just better throughput / lower latencies.
- jcelerier 6y ago
  > I hope to replace my finely tuned epoll / read/write reactor pattern inner loop with io_uring / eBPF and leave 99% of the code untouched... just better throughput / lower latencies. I hope to do an update of Asio at some point and it will start to use io_uring automatically as a backend
gens 6y ago
You are right. For most programs, these two are more work for not much payoff. Though they are great for some things.

No it will not. Two rather specialized tools to help with rather specific issues are no reason to throw out heaps and mounds of existing and perfectly working code and solutions

kevincox 6y ago

There are tons of applications that are built upon higher-level APIs that can be updated to take advantage of this automatically. Furthermore you don't need to have every single application updated to revolutionize something. The apps that need this will be quite happy to make the jump.

harporoeder 6y ago

I expect these technologies to be integrated into language runtimes and webservers such that most developers won't even know it being used similar to EPOLL today. While revolutionary is an extreme characterization the performance and improved API changes are at least non trivial.

dclusin 6y ago

There's a very niche conference called eBPF Summit* that has presentations from people at companies with first class engineering orgs talking about what they are doing with eBPF. The problems they are solving and the breadth of problems being solved are very impressive. [*] - https://ebpf.io/summit-2020/ https://ebpf.io/summit-2020/

jcelerier 6y ago
yep, at the Linux Audio Conference we have a paper for using eBPF to process network audio packets this year for instance (https://lac2020.sciencesconf.org/program https://lac2020.sciencesconf.org/program / paper is https://lac2020.sciencesconf.org/307835 https://lac2020.sciencesconf.org/307835) - presentation is tomorrow (happens online)
smartbit 6y ago
This conference was more a Cilium conference. Google will use cilium as the default cni on gke https://news.ycombinator.com/item?id=24212021 https://news.ycombinator.com/item?id=24212021 Not speaking was Kinvolk’s CTO Alban Crequy https://news.ycombinator.com/item?id=23042722 https://news.ycombinator.com/item?id=23042722

dmw_ng 6y ago

io_uring is no more specialized than an SSD is today, our storage interfaces went from handling between 1 and 4 requests in parallel for the past 50 years to suddenly handling literally hundreds on consumer grade hardware (IIRC NVMe is specced either for up to 64k queue depth or unlimited queue depth). Of course the software environment must change to keep up, is it really reasonable to continue feeding such devices one IO at a time because that's how it was done in 1970? It is not fun to program against io_uring just as BPF can be a nightmare, but that's a problem for userspace to solve with libraries and better abstractions, much in the same way userspace usually doesn't have to deal with parsing the shared ELF segment exported by the kernel (glibc does that) or writing complex routines for fetching the system time (the ELF segment does that), both of which are implementation details necessary for extracting the full performance from modern hardware. We'll catch up eventually, but first the lower level interfaces must exist. In meantime, I cringe Every. Single. Time. I run UNIX find or du over my home directory, realizing it could have completed in a fraction of the time for almost 10 years now if only our traditional software environment was awakened to the reality of the hardware it has long since run on.

erk__ 6y ago
Well at least in the near future I don't think you will see it on any other UNIX than Linux as it is compared to aio not a standard, but a Linux specific api.
- dmw_ng 6y ago
  BPF and io_uring architectural styles both have heritage in and close equivalents at least on BSD. You could consider io_uring as nothing but a slightly fancier cousin of FreeBSD's netmap that exists for exactly the same reason, they just wear different shades of lipstick. BPF of course came directly from BSD into Linux Any inevitable standard will likely come from userspace, much in the same way libpcap successfully papered over raw packet capture for a huge variety of operating systems. The underlying kernel interface is basically just details. We're decades past the point where application portability required standards like POSIX to make progress, I imagine comparatively few modern programmers even know much about those kinds of standards any more.

cpuguy83 6y ago

Neither ebpf or io_uring are "specialized", in fact they are specifically not specialized which is what makes them revolutionary (along with being better than currently specialized apis like aio).

gHosts 6y ago

What excites me is if you're working down at the high frequency, low latency I/O domain... linux traditionally has actually sort of sucked. We're _very_ good at buffering things up and handling large chunks to gain high throughput. We have layers and layers of magic that make us _very_ good at that. For cases where you have lots of devices throwing small chunks at high frequency and you have to respond (in this one, do a small computation, and out that one...) we've sucked bad. That's why bare metal RTOS's still exist. The io_uring / eBPF combo looks _very_ promising for opening up that domain in a tidy fashion. I also hope there will be no reason to throw away "heaps and mounds of existing and perfectly working code and solutions". I hope to replace my finely tuned epoll / read/write reactor pattern inner loop with io_uring / eBPF and leave 99% of the code untouched... just better throughput / lower latencies.

jcelerier 6y ago
> I hope to replace my finely tuned epoll / read/write reactor pattern inner loop with io_uring / eBPF and leave 99% of the code untouched... just better throughput / lower latencies. I hope to do an update of Asio at some point and it will start to use io_uring automatically as a backend

gens 6y ago

You are right. For most programs, these two are more work for not much payoff. Though they are great for some things.