4 ms·

You can't generally re-implement features of a kernel on another platform.

by DasIch 11y ago

pdkl95 11y ago
You usually can in any situation where you would be porting the library in question. I'm going to have a hard time if I want to make OpenGL calls work on my old Z80 micros... but why would I want to? Anywhere I would want to port a library that uses epoll it is trivial to re-implement the functionality using poll(2)/select(2) or even simply setting O_NONBLOCK and busy-waiting while manually polling read(2). Other platforms may have other non-standard features that are full or partial replacements. Performance may not be the same, but that's a common problem when porting software. I'm sure it's possible to find various pathological cases where this is difficult. In the common cases, proper abstraction is worth it simply for the complexity and maintenance costs. edit: None of this should be taken as a recommendation for the epoll interface. vezzy-fnord is right - there are much better solutions available.
vezzy-fnord 11y ago
Sure you can. It might be the case that the level of granularity is low enough that you'd need to do it in kernel/supervisor mode itself to access private data structures like the process control block, instead of doing compatibility layers, shims or daemons in user space. In most cases, the latter is fully feasible. You just generalize by categorizing the functionality of a source kernel and finding the equivalent taxonomy in a target kernel. It's not like any commodity OS kernel these days doesn't have a feature to multiplex I/O on ports, fds, handles or whatever that it's that prohibitive to shim a syscall, or more often than not, substitute it at build-time or just use an existing library/framework that encapsulates the details.
- JoshTriplett 11y ago
  > It might be the case that the level of granularity is low enough that you'd need to do it in kernel/supervisor mode itself to access private data structures like the process control block, instead of doing compatibility layers, shims or daemons in user space. Not all kernels allow you to do that. Even among those that do, you'd have to have some level of privilege to install it. (I know I'd look very skeptically at any program or library that wanted an out-of-tree kernel module installed.) And even if you manage to get that code submitted upstream, and they accept it (not at all a given), now you depend on the most recent version of that target OS, and you don't run on older versions of it. Which starts you down the same non-portable road you're complaining about. That said, I absolutely think making such changes in upstream kernels/OSes is the right approach. Make your kernel/OS better to accommodate your userspace software, rather than making your userspace software worse to accommodate your kernel/OS.
  - vezzy-fnord 11y ago
    Again, this would be more if you needed to emulate things like precise kernel thread semantics, page eviction strategies or whatnot. When talking about system calls (the overwhelming majority of cases), then those are usually well-defined, relatively high level "chunks" of resource management that either have corollaries or can be safely reimplemented in a utility library.
    - JoshTriplett 11y ago
      How would you suggest reimplementing cgroups on every other POSIX OS? In particular, the behavior where an entire tree of child processes can't escape from that group? Some kernels may have an equivalent to this (though the semantics may not match), but others don't. How about prctl's PR_SET_CHILD_SUBREAPER, which makes a process act like PID 1, as the target for reparenting child processes? Or fanotify/inotify/etc? (A polling-based interface doesn't work here: it misses things due to races, and wastes power by waking up too often.) Or mount namespaces, network namespaces, and other unshare/clone flags? You could reimplement those things in another kernel, but almost certainly not in userspace unless a feature with the same semantics already exists in the target kernel.
      - vezzy-fnord 11y ago
        unless a feature with the same semantics already exists in the target kernel The point is they do. For instance, Solaris has contracts (direct corollary to cgroups, actually predating cgroups). In Solaris, orphans are adopted by the kernel and init(1M) can be restarted upon a failure condition. Some BSD flavors have procctl(2). Polling-based interfaces can work, since a port does not have to be completely equivalent in terms of benchmarking, it must conform well by a cost-benefit analysis. File system event notifications are something many platforms reliably have, so that even devices with strict power management policy can incur them (or more often than not they'll incur the cost of polling regardless). Most Unixes have converged on rfork(2)-like primitives that can customize process contexts to share reasonably well. I'm not panning systemd for anything, mind you. I'm speaking in general terms. systemd is decidedly unportable, I'm more than well aware and don't have trouble with that aspect.
pjc50 11y ago
Oh yes you can, even if it may be ugly; I had to write a porting layer to run 'berkely sockets' networking code on a weird closed source embedded platform that wanted to do all networking through callbacks.

You can't generally re-implement features of a kernel on another platform.

by DasIch 11y ago

You can't generally re-implement features of a kernel on another platform.

You usually can in any situation where you would be porting the library in question. I'm going to have a hard time if I want to make OpenGL calls work on my old Z80 micros... but why would I want to? Anywhere I would want to port a library that uses epoll it is trivial to re-implement the functionality using poll(2)/select(2) or even simply setting O_NONBLOCK and busy-waiting while manually polling read(2). Other platforms may have other non-standard features that are full or partial replacements. Performance may not be the same, but that's a common problem when porting software. I'm sure it's possible to find various pathological cases where this is difficult. In the common cases, proper abstraction is worth it simply for the complexity and maintenance costs. edit: None of this should be taken as a recommendation for the epoll interface. vezzy-fnord is right - there are much better solutions available.

vezzy-fnord 11y ago

Sure you can. It might be the case that the level of granularity is low enough that you'd need to do it in kernel/supervisor mode itself to access private data structures like the process control block, instead of doing compatibility layers, shims or daemons in user space. In most cases, the latter is fully feasible. You just generalize by categorizing the functionality of a source kernel and finding the equivalent taxonomy in a target kernel. It's not like any commodity OS kernel these days doesn't have a feature to multiplex I/O on ports, fds, handles or whatever that it's that prohibitive to shim a syscall, or more often than not, substitute it at build-time or just use an existing library/framework that encapsulates the details.

JoshTriplett 11y ago
> It might be the case that the level of granularity is low enough that you'd need to do it in kernel/supervisor mode itself to access private data structures like the process control block, instead of doing compatibility layers, shims or daemons in user space. Not all kernels allow you to do that. Even among those that do, you'd have to have some level of privilege to install it. (I know I'd look very skeptically at any program or library that wanted an out-of-tree kernel module installed.) And even if you manage to get that code submitted upstream, and they accept it (not at all a given), now you depend on the most recent version of that target OS, and you don't run on older versions of it. Which starts you down the same non-portable road you're complaining about. That said, I absolutely think making such changes in upstream kernels/OSes is the right approach. Make your kernel/OS better to accommodate your userspace software, rather than making your userspace software worse to accommodate your kernel/OS.
- vezzy-fnord 11y ago
  Again, this would be more if you needed to emulate things like precise kernel thread semantics, page eviction strategies or whatnot. When talking about system calls (the overwhelming majority of cases), then those are usually well-defined, relatively high level "chunks" of resource management that either have corollaries or can be safely reimplemented in a utility library.
  - JoshTriplett 11y ago
    How would you suggest reimplementing cgroups on every other POSIX OS? In particular, the behavior where an entire tree of child processes can't escape from that group? Some kernels may have an equivalent to this (though the semantics may not match), but others don't. How about prctl's PR_SET_CHILD_SUBREAPER, which makes a process act like PID 1, as the target for reparenting child processes? Or fanotify/inotify/etc? (A polling-based interface doesn't work here: it misses things due to races, and wastes power by waking up too often.) Or mount namespaces, network namespaces, and other unshare/clone flags? You could reimplement those things in another kernel, but almost certainly not in userspace unless a feature with the same semantics already exists in the target kernel.
    - vezzy-fnord 11y ago
      unless a feature with the same semantics already exists in the target kernel The point is they do. For instance, Solaris has contracts (direct corollary to cgroups, actually predating cgroups). In Solaris, orphans are adopted by the kernel and init(1M) can be restarted upon a failure condition. Some BSD flavors have procctl(2). Polling-based interfaces can work, since a port does not have to be completely equivalent in terms of benchmarking, it must conform well by a cost-benefit analysis. File system event notifications are something many platforms reliably have, so that even devices with strict power management policy can incur them (or more often than not they'll incur the cost of polling regardless). Most Unixes have converged on rfork(2)-like primitives that can customize process contexts to share reasonably well. I'm not panning systemd for anything, mind you. I'm speaking in general terms. systemd is decidedly unportable, I'm more than well aware and don't have trouble with that aspect.

pjc50 11y ago

Oh yes you can, even if it may be ugly; I had to write a porting layer to run 'berkely sockets' networking code on a weird closed source embedded platform that wanted to do all networking through callbacks.