3 ms·

So the argument is that extending kqueue's interface to handle more and more event is worse than turning more and more events into xxxfd subsystems. Why is that

So the argument is that extending kqueue's interface to handle more and more event is worse than turning more and more events into xxxfd subsystems. Why is that worse, again? Like, not in "it's not a composable design" abstract sense but in concrete "you can't do this and that, which are useful things" sense?

jsnell 2y ago
Yeah, the article really doesn't make a case as much as assert it. One way to think about this is whether all these non-file fds are useful for a variety of generic operations, or whether they're only used for epoll and one-off system calls or ioctls specific to that type of fd. If it's the latter, it seems hard to believe that there actually is some kind of composability advantage. So, what can you do with them? 1. You can use these fds with poll()/select(), not just epoll. That's not a big deal, since nobody really should be using either of them. But it does also mean that you should ideally be able to use them with any future event notification syscalls as well. And the invention of new mechanisms is not hypothetical, since Linux added io_uring. I'd be curious to know how well io_uring supports arbitrary fd types, and whether they got that support "for free" or if io_uring needs to be extended for every new fd type. 2. You can typically read() structured data from those fds describing the specific event. With kqueue all that data would need to be passed through struct kevent, which needs to be fixed size and generic enough to be shared between all the different event types. 3. You can pass individual fds between processes, either via UDS or fork(). I expect you would not be able to do that for individual kqueue filters. 4. You can close() the fd to release the underlying kernel resource. This doesn't seem interesting, just listing it for completeness. So there's probably enough smoke there that one could at least argue the case. It's too bad the author didn't.
- timewizard 2y ago
  [dead]
- stevefolta 2y ago
  > ...nobody really should be using either of them. What's wrong with poll(), at least for smaller values of nfds? And what should one use instead when writing POSIX-compatible portable code, where epoll() and kqueue() don't exist?
  - somat 2y ago
    Fun fact, on openbsd(at least, I don't know about freebsd) select and poll were reworked to use kqueue internally. The thread is an interesting read as it sounds like the naive approach has negative performance implications, knowing openbsd I suspect the main motivation was to reduce maintenance burden, that is, make it easier to reason about the internals of event driven mechanisms by only having one such system to worry about. https://marc.info/?l=openbsd-tech&m=162444132017561&w=2 https://marc.info/?l=openbsd-tech&m=162444132017561&w=2
ajross 2y ago
Do you have a third party driver that exposes a file descriptor for some oddball purpose? epoll can wait on that. It's absolutely true that within the bounds of what the OS supports, both interfaces are complete. But that's sort of a specious point. One requires a giant tree of extra types be amended every time something changes, while the other exploits a nearly-five-decade old abstraction that already encapsulates the idea of "something on which you might want to wait".
- markjdb 2y ago
  The same is true of kqueue/kevent though... the driver just needs to decide which filters it wants to implement. There's no need to extend kqueue when adding some custom driver or fd type. One just needs to define some semantics for the existing filters.
  - ajross 2y ago
    > the driver just needs to decide That's pretty much the definition of technical debt though. "This interface works fine for you if you take steps to handle it specially according to its requirements". It makes kqueue into a point of friction for everything in the system wanting to provide a file descriptor-like API.
    - netbsdusers 2y ago
      No it isn't. Letting files be poll/select/epoll'd isn't free either. They don't get support for that by magic. A poll operation has to be coded, and this is just as a much a "point of friction" then as supporting kqueue. (It bears mentioning as well that on at least DragonFly BSD and OpenBSD, they have reimplemented poll()/select() to use kqueue's knotes mechanism, and so now you only have to write a VOP_KQFILTER implementation and not a VOP_POLL too.)
      - ajross 2y ago
        > Letting files be poll/select/epoll'd isn't free either. Yes, but those slashes are showing the lie in the statement. Letting files be polled/selected isn't "free", but it's standard. The poll() method has been in struct file_operations for literally decades[1]. Adding "epoll support" requires no meaningful changes to the API, for any device that ever supported select(). That kind of evolutionary flexibility (the opposite of "technical debt") is generally regarded as good design. And it's something that epoll had designed in and something that queue lacks, having decided to go its own way. And it's not unreasonable to call that out, IMHO. [1] It's present in commit 1da177e4c3f4 ("Linux-2.6.12-rc2"), which is the very first git commit. I know people maintain archives of older trees, but I'm too lazy to dig. Suffice it to say that epoll relies on an interface that is likely older than many of the driver developers using it.
Someone 2y ago
I would think it’s the other way around. If you have separate subsystems for events of types A, B, C, etc. there’s no easy way you can have a process wait for “an event of type A or C, whichever comes first”. There also is the thing that, sometimes, different types of event are identical types, lower down in the stack. A simple example are keyboard and mouse events generated from USB devices. Why would the low-level USB stack have to do a switch on device type to figure out what subsystem to post an event to?
IshKebab 2y ago
Because "wait for an event" is a good abstraction for many event types. The epoll interface is a generic wait interface whereas the kqueue one is a hardcoded list of things you can wait for. Think about async programming you have a single `await` keyword that can await lots of different things.
- toast0 2y ago
  How many event types do you need though? And would you actually need to do less work to add support for a new one to a user program? It would be different work, for sure, but I don't think it would be meaningfully less work.

jsnell 2y ago

Yeah, the article really doesn't make a case as much as assert it. One way to think about this is whether all these non-file fds are useful for a variety of generic operations, or whether they're only used for epoll and one-off system calls or ioctls specific to that type of fd. If it's the latter, it seems hard to believe that there actually is some kind of composability advantage. So, what can you do with them? 1. You can use these fds with poll()/select(), not just epoll. That's not a big deal, since nobody really should be using either of them. But it does also mean that you should ideally be able to use them with any future event notification syscalls as well. And the invention of new mechanisms is not hypothetical, since Linux added io_uring. I'd be curious to know how well io_uring supports arbitrary fd types, and whether they got that support "for free" or if io_uring needs to be extended for every new fd type. 2. You can typically read() structured data from those fds describing the specific event. With kqueue all that data would need to be passed through struct kevent, which needs to be fixed size and generic enough to be shared between all the different event types. 3. You can pass individual fds between processes, either via UDS or fork(). I expect you would not be able to do that for individual kqueue filters. 4. You can close() the fd to release the underlying kernel resource. This doesn't seem interesting, just listing it for completeness. So there's probably enough smoke there that one could at least argue the case. It's too bad the author didn't.

timewizard 2y ago
[dead]
stevefolta 2y ago
> ...nobody really should be using either of them. What's wrong with poll(), at least for smaller values of nfds? And what should one use instead when writing POSIX-compatible portable code, where epoll() and kqueue() don't exist?
- somat 2y ago
  Fun fact, on openbsd(at least, I don't know about freebsd) select and poll were reworked to use kqueue internally. The thread is an interesting read as it sounds like the naive approach has negative performance implications, knowing openbsd I suspect the main motivation was to reduce maintenance burden, that is, make it easier to reason about the internals of event driven mechanisms by only having one such system to worry about. https://marc.info/?l=openbsd-tech&m=162444132017561&w=2 https://marc.info/?l=openbsd-tech&m=162444132017561&w=2

ajross 2y ago

Do you have a third party driver that exposes a file descriptor for some oddball purpose? epoll can wait on that. It's absolutely true that within the bounds of what the OS supports, both interfaces are complete. But that's sort of a specious point. One requires a giant tree of extra types be amended every time something changes, while the other exploits a nearly-five-decade old abstraction that already encapsulates the idea of "something on which you might want to wait".

markjdb 2y ago
The same is true of kqueue/kevent though... the driver just needs to decide which filters it wants to implement. There's no need to extend kqueue when adding some custom driver or fd type. One just needs to define some semantics for the existing filters.
- ajross 2y ago
  > the driver just needs to decide That's pretty much the definition of technical debt though. "This interface works fine for you if you take steps to handle it specially according to its requirements". It makes kqueue into a point of friction for everything in the system wanting to provide a file descriptor-like API.
  - netbsdusers 2y ago
    No it isn't. Letting files be poll/select/epoll'd isn't free either. They don't get support for that by magic. A poll operation has to be coded, and this is just as a much a "point of friction" then as supporting kqueue. (It bears mentioning as well that on at least DragonFly BSD and OpenBSD, they have reimplemented poll()/select() to use kqueue's knotes mechanism, and so now you only have to write a VOP_KQFILTER implementation and not a VOP_POLL too.)
    - ajross 2y ago
      > Letting files be poll/select/epoll'd isn't free either. Yes, but those slashes are showing the lie in the statement. Letting files be polled/selected isn't "free", but it's standard. The poll() method has been in struct file_operations for literally decades[1]. Adding "epoll support" requires no meaningful changes to the API, for any device that ever supported select(). That kind of evolutionary flexibility (the opposite of "technical debt") is generally regarded as good design. And it's something that epoll had designed in and something that queue lacks, having decided to go its own way. And it's not unreasonable to call that out, IMHO. [1] It's present in commit 1da177e4c3f4 ("Linux-2.6.12-rc2"), which is the very first git commit. I know people maintain archives of older trees, but I'm too lazy to dig. Suffice it to say that epoll relies on an interface that is likely older than many of the driver developers using it.

Someone 2y ago

I would think it’s the other way around. If you have separate subsystems for events of types A, B, C, etc. there’s no easy way you can have a process wait for “an event of type A or C, whichever comes first”. There also is the thing that, sometimes, different types of event are identical types, lower down in the stack. A simple example are keyboard and mouse events generated from USB devices. Why would the low-level USB stack have to do a switch on device type to figure out what subsystem to post an event to?

IshKebab 2y ago

Because "wait for an event" is a good abstraction for many event types. The epoll interface is a generic wait interface whereas the kqueue one is a hardcoded list of things you can wait for. Think about async programming you have a single `await` keyword that can await lots of different things.

toast0 2y ago
How many event types do you need though? And would you actually need to do less work to add support for a new one to a user program? It would be different work, for sure, but I don't think it would be meaningfully less work.