3 ms·

I generally agree with your comment, especially coming from OpenBSD where /proc does not exist, for reasons you mentioned, as well as a purely practical one: le

I generally agree with your comment, especially coming from OpenBSD where /proc does not exist, for reasons you mentioned, as well as a purely practical one: let's not pressure the VFS for no reason. I think the biggest issue with /proc at the moment is that it leaks in core Linux components, and getting rid of it is becoming impossible. Some years ago I worked on adding `$ORIGIN` rpath support in OpenBSD and looked at how Linux did it. Mind you, the dynamic linker (ldlinux.so) grabs the running executable path from /proc... One thing though, you focus a lot on syscalls, but I think the challenge in recent years has been to find new ways to have kernel <-> userspace interactions _outside_ of syscall, which are cumbersome to use, rigid in structure, and practically speaking, there are only so many entries you can store in an IDT. The work on netlink is going in the right direction IMHO. It allows userland to communicate bidirectionally with the kernel, register on specific events, etc; while using a familiar tooling with bsd sockets that is convenient from both shell and programming languages.

o11c 3y ago
> Some years ago I worked on adding `$ORIGIN` rpath support in OpenBSD and looked at how Linux did it. Mind you, the dynamic linker (ldlinux.so) grabs the running executable path from /proc... `getauxval(AT_EXECFN)` is "better" but there are weird edge cases with `fexecve` where you only get `AT_EXECFD`, even ignoring the inevitable TOCTOU.
- Galanwe 3y ago
  I am a bit blurry on the details because I worked on that almost 10 years ago to support a program that used (and abused) `$ORIGIN`. I ended up pushing the executable filename through the aux vector to the dynamic linker, which is simar to what you mention AFAIU. Never managed to get that merged upstream though. Glad to know there's now a _standard_ way.
twic 3y ago
> One thing though, you focus a lot on syscalls, but I think the challenge in recent years has been to find new ways to have kernel <-> userspace interactions _outside_ of syscall, which are cumbersome to use, rigid in structure, and practically speaking, there are only so many entries you can store in an IDT. There's always ioctls. You could have something a lot like /proc and /sys, with a directory hierarchy modelling various interesting things, but rather than the files having textual contents, you interact with them via ioctls. ioctls are identified by a device-specific integer, so you have masses of namespace (about 2*32 operations per device type, and you can have a lot of device types). They take or receive a block of memory in one go, so there is no parsing or formatting or worrying about inconsistent reads. I think this is even simpler than protocol-esque interfaces like netlink.
- deleted 3y ago
  [deleted]
- Galanwe 3y ago
  Agree, ioctls are a good replacement of /proc for exposing simple, structured information to/from the kernel. It is still not perfect for all use cases though, especially because it forces you to operate in a _polling_ fashion. Programs like top/iotop/htop are a good example (also, who is not guilty of abusing one liners a-la `watch -n1 cat /proc/meminfo`?). Instead of polling /proc or ioctls, you would really like to register on events, and get notifications pushed. Netlink is well suited for these kinds of scenarios, though mostly to get notified of network interface changes last I checked.
  - twic 3y ago
    That's an interesting point. I wonder if some sort of polling interface to ioctls would be feasible. On top of kqueue or io_uring or something maybe.
  - zokier 3y ago
    Netlink is pretty much as good as it gets here. It is explicitly designed as improvement over ioctl mess > Netlink is often described as an ioctl() replacement. It aims to replace fixed-format C structures as supplied to ioctl() with a format which allows an easy way to add or extended the arguments.
- crabbone 3y ago
  > There's always ioctls. Not good. Not so long ago I ran into a problem with mdadm which instead of reading sysfs used ioctls which simply didn't carry enough information for it to function properly. So, it still has bugs because of that, but it being written in C, reading from a file and parsing stuff from a string seems to cause a lot of melancholy in its maintainers, so the bug has been on slow burner for years now.
io_urine 3y ago
> and practically speaking, there are only so many entries you can store in an IDT. The number of syscalls isn't limited by this, they all go through the same interrupt and are dispatched to a specific handler function based on the value of a register. For example this is how it works on arm64: https://elixir.bootlin.com/linux/v6.5/source/arch/arm64/kernel/syscall.c#L139 https://elixir.bootlin.com/linux/v6.5/source/arch/arm64/kern... And on x86: https://elixir.bootlin.com/linux/v6.5/source/arch/x86/entry/common.c#L50 https://elixir.bootlin.com/linux/v6.5/source/arch/x86/entry/...
crabbone 3y ago
What about stuff like udev? Or anything that has a kernel module and a controlling program running in userspace, eg. iscisiadm, mdadm etc? This way any given utility is free to implement and experiment with the communication protocol however they want. But, more generally, if we want general-purpose communication, why not press further and have relational database-like interface, with transactional semantics, triggers, fine-grained ownership?..
goodthenandnow 3y ago
> One thing though, you focus a lot on syscalls, but I think the challenge in recent years has been to find new ways to have kernel <-> userspace interactions _outside_ of syscall, which are cumbersome to use, rigid in structure, and practically speaking, there are only so many entries you can store in an IDT. Today there is another option for doing the kernel-userspace communication: using an io_uring like interface. Yes, it's a syscall, but it's one with enough extensibility and performance characteristics. It's extensible because it's close to an IPC with the kernel (and it could be changed to be a new IPC mechanism for process-to-process communication if it isn't already). And it's performant because it's based in shared memory and is assyncronous by default.

o11c 3y ago

> Some years ago I worked on adding `$ORIGIN` rpath support in OpenBSD and looked at how Linux did it. Mind you, the dynamic linker (ldlinux.so) grabs the running executable path from /proc... `getauxval(AT_EXECFN)` is "better" but there are weird edge cases with `fexecve` where you only get `AT_EXECFD`, even ignoring the inevitable TOCTOU.

Galanwe 3y ago
I am a bit blurry on the details because I worked on that almost 10 years ago to support a program that used (and abused) `$ORIGIN`. I ended up pushing the executable filename through the aux vector to the dynamic linker, which is simar to what you mention AFAIU. Never managed to get that merged upstream though. Glad to know there's now a _standard_ way.

twic 3y ago

> One thing though, you focus a lot on syscalls, but I think the challenge in recent years has been to find new ways to have kernel <-> userspace interactions _outside_ of syscall, which are cumbersome to use, rigid in structure, and practically speaking, there are only so many entries you can store in an IDT. There's always ioctls. You could have something a lot like /proc and /sys, with a directory hierarchy modelling various interesting things, but rather than the files having textual contents, you interact with them via ioctls. ioctls are identified by a device-specific integer, so you have masses of namespace (about 2*32 operations per device type, and you can have a lot of device types). They take or receive a block of memory in one go, so there is no parsing or formatting or worrying about inconsistent reads. I think this is even simpler than protocol-esque interfaces like netlink.

deleted 3y ago
[deleted]
Galanwe 3y ago
Agree, ioctls are a good replacement of /proc for exposing simple, structured information to/from the kernel. It is still not perfect for all use cases though, especially because it forces you to operate in a _polling_ fashion. Programs like top/iotop/htop are a good example (also, who is not guilty of abusing one liners a-la `watch -n1 cat /proc/meminfo`?). Instead of polling /proc or ioctls, you would really like to register on events, and get notifications pushed. Netlink is well suited for these kinds of scenarios, though mostly to get notified of network interface changes last I checked.
- twic 3y ago
  That's an interesting point. I wonder if some sort of polling interface to ioctls would be feasible. On top of kqueue or io_uring or something maybe.
- zokier 3y ago
  Netlink is pretty much as good as it gets here. It is explicitly designed as improvement over ioctl mess > Netlink is often described as an ioctl() replacement. It aims to replace fixed-format C structures as supplied to ioctl() with a format which allows an easy way to add or extended the arguments.
crabbone 3y ago
> There's always ioctls. Not good. Not so long ago I ran into a problem with mdadm which instead of reading sysfs used ioctls which simply didn't carry enough information for it to function properly. So, it still has bugs because of that, but it being written in C, reading from a file and parsing stuff from a string seems to cause a lot of melancholy in its maintainers, so the bug has been on slow burner for years now.

io_urine 3y ago

> and practically speaking, there are only so many entries you can store in an IDT. The number of syscalls isn't limited by this, they all go through the same interrupt and are dispatched to a specific handler function based on the value of a register. For example this is how it works on arm64: https://elixir.bootlin.com/linux/v6.5/source/arch/arm64/kernel/syscall.c#L139 https://elixir.bootlin.com/linux/v6.5/source/arch/arm64/kern... And on x86: https://elixir.bootlin.com/linux/v6.5/source/arch/x86/entry/common.c#L50 https://elixir.bootlin.com/linux/v6.5/source/arch/x86/entry/...

crabbone 3y ago

What about stuff like udev? Or anything that has a kernel module and a controlling program running in userspace, eg. iscisiadm, mdadm etc? This way any given utility is free to implement and experiment with the communication protocol however they want. But, more generally, if we want general-purpose communication, why not press further and have relational database-like interface, with transactional semantics, triggers, fine-grained ownership?..

goodthenandnow 3y ago

> One thing though, you focus a lot on syscalls, but I think the challenge in recent years has been to find new ways to have kernel <-> userspace interactions _outside_ of syscall, which are cumbersome to use, rigid in structure, and practically speaking, there are only so many entries you can store in an IDT. Today there is another option for doing the kernel-userspace communication: using an io_uring like interface. Yes, it's a syscall, but it's one with enough extensibility and performance characteristics. It's extensible because it's close to an IPC with the kernel (and it could be changed to be a new IPC mechanism for process-to-process communication if it isn't already). And it's performant because it's based in shared memory and is assyncronous by default.