5 ms·

I am confused about the fact that this is "something that allows us access to all the Linux-specific bits, instead of limiting itself to the least common denomi

I am confused about the fact that this is "something that allows us access to all the Linux-specific bits, instead of limiting itself to the least common denominator of UNIX." Doesn't that make anything written with this incompatible with, say, FreeBSD or MacOS?

kragniz 11y ago
It does. He's saying that instead of writing a library that abstracts using the least common denominator across all kernels, it only cares about Linux. Doing this allows you to make use of Linux-specific APIs (for example cgroups elsewhere in systemd).
- pdkl95 11y ago
  > abstract Leaking the lower-level interface is a terrible "abstraction". > least common denominator across all kernel That's one approach. A better approach is to stop leaking the lower level interfaces and providing the higher-level API you want in a way that isolates you from the details. When porting to another platform, this may require re-implementing features that are not available on the new platform. This is one of the reasons porting can be difficult. > allows you to make use of Linux-specific APIs We never had a problem using those features in the past. This is one of the reasons autotools/cmake exists.
  - DasIch 11y ago
    You can't generally re-implement features of a kernel on another platform.
    - pdkl95 11y ago
      You usually can in any situation where you would be porting the library in question. I'm going to have a hard time if I want to make OpenGL calls work on my old Z80 micros... but why would I want to? Anywhere I would want to port a library that uses epoll it is trivial to re-implement the functionality using poll(2)/select(2) or even simply setting O_NONBLOCK and busy-waiting while manually polling read(2). Other platforms may have other non-standard features that are full or partial replacements. Performance may not be the same, but that's a common problem when porting software. I'm sure it's possible to find various pathological cases where this is difficult. In the common cases, proper abstraction is worth it simply for the complexity and maintenance costs. edit: None of this should be taken as a recommendation for the epoll interface. vezzy-fnord is right - there are much better solutions available.
    - vezzy-fnord 11y ago
      Sure you can. It might be the case that the level of granularity is low enough that you'd need to do it in kernel/supervisor mode itself to access private data structures like the process control block, instead of doing compatibility layers, shims or daemons in user space. In most cases, the latter is fully feasible. You just generalize by categorizing the functionality of a source kernel and finding the equivalent taxonomy in a target kernel. It's not like any commodity OS kernel these days doesn't have a feature to multiplex I/O on ports, fds, handles or whatever that it's that prohibitive to shim a syscall, or more often than not, substitute it at build-time or just use an existing library/framework that encapsulates the details.
      - JoshTriplett 11y ago
        > It might be the case that the level of granularity is low enough that you'd need to do it in kernel/supervisor mode itself to access private data structures like the process control block, instead of doing compatibility layers, shims or daemons in user space. Not all kernels allow you to do that. Even among those that do, you'd have to have some level of privilege to install it. (I know I'd look very skeptically at any program or library that wanted an out-of-tree kernel module installed.) And even if you manage to get that code submitted upstream, and they accept it (not at all a given), now you depend on the most recent version of that target OS, and you don't run on older versions of it. Which starts you down the same non-portable road you're complaining about. That said, I absolutely think making such changes in upstream kernels/OSes is the right approach. Make your kernel/OS better to accommodate your userspace software, rather than making your userspace software worse to accommodate your kernel/OS.
        vezzy-fnord 11y ago
        Again, this would be more if you needed to emulate things like precise kernel thread semantics, page eviction strategies or whatnot. When talking about system calls (the overwhelming majority of cases), then those are usually well-defined, relatively high level "chunks" of resource management that either have corollaries or can be safely reimplemented in a utility library.
    - pjc50 11y ago
      Oh yes you can, even if it may be ugly; I had to write a porting layer to run 'berkely sockets' networking code on a weird closed source embedded platform that wanted to do all networking through callbacks.
msbarnett 11y ago
The systemd project doesn't value portability at all and everything they release is basically tied to Linux only
- jimktrains2 11y ago
  Didn't they do that with cgroups already?
  - msbarnett 11y ago
    there are many things in systemd tied to Linux-only APIs. This isn't the first, and cgroups is just one example.
- 0xFFC 11y ago
  I think personally (I may be wrong , please correct me) this was a huge push from Redhat , because they want a Workstation , and with old design (having SysVInit with whole bunch of small and separated stuff) and without launchd clone in Linux world they think they cannot achieve that goal.
  - vezzy-fnord 11y ago
    Ironically enough, sysvinit and sysv-rc weren't particularly good on the whole "small and separated" aspect [1]. This has been known for a while, and a lot of work has been done to address the problem domain, much of it understudied and not well known [2]. I don't think Red Hat had all that much of a stake initially. Kay Sievers was actually a SUSE employee at the time systemd was conceived for public release, IIRC. Red Hat's interests seem far more directed at cloud and containers (see: Project Atomic) than workstations, and indeed the direction systemd has been heading reflects that. [1] http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/system-5-rc-problems.html http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/sy... [2] http://blog.darknedgy.net/technology/2015/09/05/0/ http://blog.darknedgy.net/technology/2015/09/05/0/
jepler 11y ago
nonportability is typically a design goal of systemd. See http://0pointer.de/blog/projects/the-biggest-myths.html http://0pointer.de/blog/projects/the-biggest-myths.html down at point 16 ("Myth: systemd is not portable for no reason.")
- DasIch 11y ago
  Nonportability isn't a goal, portability though isn't. Portability isn't all that great a goal to begin with. You limit yourself to a lowest common denominator and sometimes you need to make compromises between platforms. This makes it very hard to impossible to do truly innovative and good things. GUIs and the web in particular are the perfect example. Every GUI that's not specifically designed for a platform is always worse than the native solution. Additionally the web is laughably behind. Trivial WebGL experiments that you could do natively decades ago regularly hit the HN frontpage. On a lower level you have C which is bad for many reasons but one of them is trying to be portable and so you end up with not being able to assume 2's complement. Whenever you can do without portability, you're probably better off ignoring it.
  - vezzy-fnord 11y ago
    The web is laughably behind not because of portability concerns, but because it fundamentally began as a poor man's hypertext system that could be hacked up in two weeks for linking together CERN's document archives. Web browsers are an interesting experiment in that they're virtual machines that began from a mostly blank state independent of their host, and progressively had to rediscover decades of CS research (often haphazardly) to get them in their current state. That said, nothing truly innovative is being done with systemd to begin with.
SixSigma 11y ago
If you’re willing to restrict the flexibility of your approach, you can almost always do something better. — John Carmack
pdkl95 11y ago
This is yet another leaky abstraction. The non-portability is mentioned: If portability is essential to you, sd-event is not your best option. sd-event is a wrapper around Linux-specific APIs ... Also, "allows us to access"? Why the implication that epoll and timerfd were inaccessible?
- ori_b 11y ago
  > Why the implication that epoll and timerfd were inaccessible? With high level, portable, abstractions, it may very well be impossible to expect the event loop to know how to deal sensibly with, eg, timerfd.
  - Sanddancer 11y ago
    libuv seems to handle such things pretty well. Ported to AIX, *BSD, Darwin, Solaris, and Windows.
nitrogen 11y ago
Indeed, that paragraph seems to say that they didn't reuse the existing libevent or libev because they were too portable.
- gaius 11y ago
  Exactly, how to do you embrace, extend, extinguish if you're writing portable code?
  - teddyh 11y ago
    The first step is “embrace”. Writing kernel-specific code is hardly “embracing”.
    - Sanddancer 11y ago
      This is the extend step. They've embraced the unix philosophy, and are now trying to make it "better", throwing compatibility with other systems out the window for little reason.
ori_b 11y ago
Yes. Their response is "Who cares? We're targeting Linux. If you want BSD, you know where to find it."
- epoch1970 11y ago
  Many members of the BSD communities are quite happy with this state of affairs. This is especially true for the BSD users who became BSD users after experiencing severe problems with systemd and Linux. The very last thing they want to see now is systemd expanding beyond Linux distros.
- vezzy-fnord 11y ago
  Nope, their response is "BSD is irrelevant. Stop using it. Having only one platform is a reason to party!" [1] It's quite hostile. [1] https://lwn.net/Articles/430699/ https://lwn.net/Articles/430699/

kragniz 11y ago

It does. He's saying that instead of writing a library that abstracts using the least common denominator across all kernels, it only cares about Linux. Doing this allows you to make use of Linux-specific APIs (for example cgroups elsewhere in systemd).

pdkl95 11y ago
> abstract Leaking the lower-level interface is a terrible "abstraction". > least common denominator across all kernel That's one approach. A better approach is to stop leaking the lower level interfaces and providing the higher-level API you want in a way that isolates you from the details. When porting to another platform, this may require re-implementing features that are not available on the new platform. This is one of the reasons porting can be difficult. > allows you to make use of Linux-specific APIs We never had a problem using those features in the past. This is one of the reasons autotools/cmake exists.
- DasIch 11y ago
  You can't generally re-implement features of a kernel on another platform.
  - pdkl95 11y ago
    You usually can in any situation where you would be porting the library in question. I'm going to have a hard time if I want to make OpenGL calls work on my old Z80 micros... but why would I want to? Anywhere I would want to port a library that uses epoll it is trivial to re-implement the functionality using poll(2)/select(2) or even simply setting O_NONBLOCK and busy-waiting while manually polling read(2). Other platforms may have other non-standard features that are full or partial replacements. Performance may not be the same, but that's a common problem when porting software. I'm sure it's possible to find various pathological cases where this is difficult. In the common cases, proper abstraction is worth it simply for the complexity and maintenance costs. edit: None of this should be taken as a recommendation for the epoll interface. vezzy-fnord is right - there are much better solutions available.
  - vezzy-fnord 11y ago
    Sure you can. It might be the case that the level of granularity is low enough that you'd need to do it in kernel/supervisor mode itself to access private data structures like the process control block, instead of doing compatibility layers, shims or daemons in user space. In most cases, the latter is fully feasible. You just generalize by categorizing the functionality of a source kernel and finding the equivalent taxonomy in a target kernel. It's not like any commodity OS kernel these days doesn't have a feature to multiplex I/O on ports, fds, handles or whatever that it's that prohibitive to shim a syscall, or more often than not, substitute it at build-time or just use an existing library/framework that encapsulates the details.
    - JoshTriplett 11y ago
      > It might be the case that the level of granularity is low enough that you'd need to do it in kernel/supervisor mode itself to access private data structures like the process control block, instead of doing compatibility layers, shims or daemons in user space. Not all kernels allow you to do that. Even among those that do, you'd have to have some level of privilege to install it. (I know I'd look very skeptically at any program or library that wanted an out-of-tree kernel module installed.) And even if you manage to get that code submitted upstream, and they accept it (not at all a given), now you depend on the most recent version of that target OS, and you don't run on older versions of it. Which starts you down the same non-portable road you're complaining about. That said, I absolutely think making such changes in upstream kernels/OSes is the right approach. Make your kernel/OS better to accommodate your userspace software, rather than making your userspace software worse to accommodate your kernel/OS.
      - vezzy-fnord 11y ago
        Again, this would be more if you needed to emulate things like precise kernel thread semantics, page eviction strategies or whatnot. When talking about system calls (the overwhelming majority of cases), then those are usually well-defined, relatively high level "chunks" of resource management that either have corollaries or can be safely reimplemented in a utility library.
  - pjc50 11y ago
    Oh yes you can, even if it may be ugly; I had to write a porting layer to run 'berkely sockets' networking code on a weird closed source embedded platform that wanted to do all networking through callbacks.

msbarnett 11y ago

The systemd project doesn't value portability at all and everything they release is basically tied to Linux only

jimktrains2 11y ago
Didn't they do that with cgroups already?
- msbarnett 11y ago
  there are many things in systemd tied to Linux-only APIs. This isn't the first, and cgroups is just one example.
0xFFC 11y ago
I think personally (I may be wrong , please correct me) this was a huge push from Redhat , because they want a Workstation , and with old design (having SysVInit with whole bunch of small and separated stuff) and without launchd clone in Linux world they think they cannot achieve that goal.
- vezzy-fnord 11y ago
  Ironically enough, sysvinit and sysv-rc weren't particularly good on the whole "small and separated" aspect [1]. This has been known for a while, and a lot of work has been done to address the problem domain, much of it understudied and not well known [2]. I don't think Red Hat had all that much of a stake initially. Kay Sievers was actually a SUSE employee at the time systemd was conceived for public release, IIRC. Red Hat's interests seem far more directed at cloud and containers (see: Project Atomic) than workstations, and indeed the direction systemd has been heading reflects that. [1] http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/system-5-rc-problems.html http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/sy... [2] http://blog.darknedgy.net/technology/2015/09/05/0/ http://blog.darknedgy.net/technology/2015/09/05/0/

jepler 11y ago

nonportability is typically a design goal of systemd. See http://0pointer.de/blog/projects/the-biggest-myths.html http://0pointer.de/blog/projects/the-biggest-myths.html down at point 16 ("Myth: systemd is not portable for no reason.")

DasIch 11y ago
Nonportability isn't a goal, portability though isn't. Portability isn't all that great a goal to begin with. You limit yourself to a lowest common denominator and sometimes you need to make compromises between platforms. This makes it very hard to impossible to do truly innovative and good things. GUIs and the web in particular are the perfect example. Every GUI that's not specifically designed for a platform is always worse than the native solution. Additionally the web is laughably behind. Trivial WebGL experiments that you could do natively decades ago regularly hit the HN frontpage. On a lower level you have C which is bad for many reasons but one of them is trying to be portable and so you end up with not being able to assume 2's complement. Whenever you can do without portability, you're probably better off ignoring it.
- vezzy-fnord 11y ago
  The web is laughably behind not because of portability concerns, but because it fundamentally began as a poor man's hypertext system that could be hacked up in two weeks for linking together CERN's document archives. Web browsers are an interesting experiment in that they're virtual machines that began from a mostly blank state independent of their host, and progressively had to rediscover decades of CS research (often haphazardly) to get them in their current state. That said, nothing truly innovative is being done with systemd to begin with.

SixSigma 11y ago

If you’re willing to restrict the flexibility of your approach, you can almost always do something better. — John Carmack

pdkl95 11y ago

This is yet another leaky abstraction. The non-portability is mentioned: If portability is essential to you, sd-event is not your best option. sd-event is a wrapper around Linux-specific APIs ... Also, "allows us to access"? Why the implication that epoll and timerfd were inaccessible?

ori_b 11y ago
> Why the implication that epoll and timerfd were inaccessible? With high level, portable, abstractions, it may very well be impossible to expect the event loop to know how to deal sensibly with, eg, timerfd.
- Sanddancer 11y ago
  libuv seems to handle such things pretty well. Ported to AIX, *BSD, Darwin, Solaris, and Windows.

nitrogen 11y ago

Indeed, that paragraph seems to say that they didn't reuse the existing libevent or libev because they were too portable.

gaius 11y ago
Exactly, how to do you embrace, extend, extinguish if you're writing portable code?
- teddyh 11y ago
  The first step is “embrace”. Writing kernel-specific code is hardly “embracing”.
  - Sanddancer 11y ago
    This is the extend step. They've embraced the unix philosophy, and are now trying to make it "better", throwing compatibility with other systems out the window for little reason.

ori_b 11y ago

Yes. Their response is "Who cares? We're targeting Linux. If you want BSD, you know where to find it."

epoch1970 11y ago
Many members of the BSD communities are quite happy with this state of affairs. This is especially true for the BSD users who became BSD users after experiencing severe problems with systemd and Linux. The very last thing they want to see now is systemd expanding beyond Linux distros.
vezzy-fnord 11y ago
Nope, their response is "BSD is irrelevant. Stop using it. Having only one platform is a reason to party!" [1] It's quite hostile. [1] https://lwn.net/Articles/430699/ https://lwn.net/Articles/430699/