9 ms·

Debian GNU/Hurd Status Update [pdf]

121 points by Tsiolkovsky 11y ago

shadeless 11y ago
Link to the talk: http://gensho.acc.umu.se/pub/debian-meetings/2015/debconf15/Debian_GNUHurd_status_update.webm http://gensho.acc.umu.se/pub/debian-meetings/2015/debconf15/... Other DebConf15 videos: http://ftp.acc.umu.se/pub/debian-meetings/2015/debconf15/ http://ftp.acc.umu.se/pub/debian-meetings/2015/debconf15/
jfaucett 11y ago
the last document linked to at the end is a really good read too - http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.37.9653&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.37...., Also I agree with him on the fdisk/mke2fs, I've often asked myself the exact same question. aparently, its not a so popular opinion though, since they're always tucked away in sbin...
- viraptor 11y ago
  Not all linux systems do that. Arch symlinks everything to /usr/bin, I believe.
mindcrime 11y ago
It's awesome to see Hurd continuing to progress. It doesn't matter that "Hurd isn't ready for mainstream use" or any of that stuff. Even if it never becomes "ready for mainstream use" it's valuable to have people experimenting and researching alternative approaches. This and Plan9 both still play important roles, and I expect that, at a minimum, developments coming out of one or both projects will continue to influence Linux and other "mainstream" OS's. Besides, it's just plain fun to play around with operating systems. I think I have an extra box laying around here idle that would be a good place to experiment with Hurd a bit.
vezzy-fnord 11y ago
The preliminary rump kernel support that arrived just a bit more than a week ago is quite promising, if still quite rudimentary. That said, a patched MPlayer linking to rump libraries has been able to play OGG files on a Hurd system: https://lists.gnu.org/archive/html/bug-hurd/2015-08/msg00027.html https://lists.gnu.org/archive/html/bug-hurd/2015-08/msg00027... Though obviously in the very initial stages, the implications of this development taken to its end could be enormous. Have the GNU Hurd implement translator or server interfaces to the rump drivers, plug it on top of an immutable package and configuration management suite like Guix (actually already being done), and for the first time in history you have a complete, general-purpose microkernel-based Unix-compatible (but beyond plain Unix) system that could rival the likes of GNU/Linux. The great thing about it is that the Hurd isn't afraid to break POSIX semantics where beneficial. This makes it have, among other things, processes running under multiple uids, true Plan 9-style namespacing, unprivileged mounts, token-based authentication, and so forth. I've been playing with it on-and-off (including cross-toolchains) and been pondering about Mach kernel emulation on Linux. A pure userland server would be most desirable, but faces real changes with modeling port rights as file descriptors sent over ancillary data, how to reliably handle user-level page fault handling on a per-application basis, mapping VM regions across address space boundaries and properly emulating RPC. Kip Macy did successful work on a port of OSF Mach from the MkLinux sources as a FreeBSD kernel module for use with launchd, XPC and other OS X modules, so I think that might end up being a concession I'll look into taking one day. I'm occupied with other projects in the meantime. It'd be also funny to have Linux "eat" itself in such a manner by becoming a host for the Hurd. Poetic justice, I suppose. But I'll wait to see if the rump kernel stuff pans out first. (MINIX 3 is great too, much better microkernel, but it's more meant for reliability rather than exposing end-user flexibility like Hurd does with translators. It also doesn't come close to the ~81% of Debian building as is with Hurd, and the MINIX devs themselves are positioning it as an embedded platform rather than general-purpose.)
- nextos 11y ago
  Guix is becoming increasingly ready for prime time. There's one big HPC install in Germany already running it. I think it's past the toy stage. I guess Guix + Hurd is a great setup, with innovative features in the kernel and userland. Both very pure in their respective ways, with Guix having functional package management, and Hurd keeping the kernel minimal so that crashes or security breaches are not catastrophic.
VLM 11y ago
The "top dumb issues" page for porters has value even for non-hurd users. In terms of "don't do this". Its on page 13.
jasonoliveira 11y ago
● Top dumb issues ● Not linux or BSD? #include <windows.h> can someone explain the above to me? the slides didn't make sense.
- koenigdavidmj 11y ago
  Poorly written preprocessor bits in C code that assume that anything anyone ever runs is either Linux, BSD (including OS X), or Windows. They basically write the following: #ifdef LINUX // include Linux headers #elif defined(BSD) // include BSD headers #else // include Windows headers. What else would it possibly be? #endif The "mach.h" one is similar: people make a bad assumption that OS X is the only Mach kernel people use. (Hurd is obviously a counterpoint, just an unpopular one.)
  - ianremsen 11y ago
    I think it's wrong to call it poorly written because the programmer only accounted their C preprocessor code for ~99.99% of desktop and mobile operating systems. The diminishing return for avoiding this approach is so impalpable it hurts. Windows, OSX/iOS/BSD, and Linux/Android literally are everything anyone ever uses, within rounding error.
    - ori_b 11y ago
      The diminishing return approach is this: #ifdef LINUX // include Linux headers #elif defined(BSD) // include BSD headers #elif defined(WINDOWS) // include Windows headers. # else #error "unsupported system" #endif
    - viraptor 11y ago
      If you develop for different platforms then it may actually make sense to do a "if linux / elif bsd / elif windows / else #error unsupported". At least it gives you a proper answer in case you forgot you have some wild crosscompiler in your path. But I understand it's not a popular concern...
ai_ja_nai 11y ago
>Hardware support >● i686 >● start of 64bit support WTF? They started 30 years ago and still on an 32bit architecture nobody cares anymore? Beside that, I really don't get the point of all this, seriously. Not from a technological benefit, be warned, but from a user perspective the benefit is almost intangible. Why should the average sysadmin care? Linux is a rock solid kernel that does everything, what is the niche that Hurd is filling?
- soong 11y ago
  >>Beside that, I really don't get the point of all this, seriously. Not from a technological benefit, be warned, but from a user perspective the benefit is almost intangible. Why should the average sysadmin care? Linux is a rock solid kernel that does everything, what is the niche that Hurd is filling? Well, that's debatable. Linux just crossed the 20 MLOCs bar. Granted, this is mostly driver code which you will only need a small subset of, but still: Linux is a huge complex blob. Linus himself said it's bloated years ago[1] and that it has a high entry barrier for new developers due to its complexity[can't find the link right now]. My guess would be that this is partly due to its classic monolithic architecture and partly due to their developement model[2]. After working with Plan9 for a while I can't help but take my hat off to the beauty and simplicity of the system. Especially as a sysadmin your life would be so much easier. No LDAP, no Kerberos, no NFS, no need to update or deploy software on/to terminals. But as it stands we're living in a land where every computer thinks it is a Mainframe. Linux has done a great deal for OSS and will without question be around for quite some time but you have to wonder how computing would look like today, if history had taken a few diffenent turns. FWIW: Is it just me, or is OS R&D gaining a little more traction lately after it almost stagnated in the 2000s ? [1] http://www.cnet.com/news/linus-torvalds-linux-is-bloated/ http://www.cnet.com/news/linus-torvalds-linux-is-bloated/ [2] https://en.wikipedia.org/wiki/Criticism_of_Linux#Kernel_code_quality https://en.wikipedia.org/wiki/Criticism_of_Linux#Kernel_code...
  - viraptor 11y ago
    I think there was some serious experimentation in 2000s, just not in the area of popular systems. Plan9 started to hit the news properly around then. A few OSes in managed languages were developed: MS Singularity, and smaller experiments like JNode and SharpOS.
viraptor 11y ago
I was hoping they will mention something about crosshurd, but there was nothing, and crosshurd doesn't work on recent debian anymore. I'd love to see some way to install from an existing system.
- vezzy-fnord 11y ago
  You can either follow my guide [1] based on updated cross-gnu scripts, or try out gnuxc [2] for bootstrapping a full Hurd distribution, with the caveat that it requires your host to be Fedora. [1] http://blog.darknedgy.net/technology/2015/07/25/0/ http://blog.darknedgy.net/technology/2015/07/25/0/ [2] https://github.com/dm0-/gnuxc https://github.com/dm0-/gnuxc
  - viraptor 11y ago
    I found this bit in your post interesting: "kdbus (which was called “neutered Mach IPC” by Neal Walfield)" - does that mean mach ipc provides typed interface registration / binding between userland processes? I admit, I know very little about mach and thought it was for kernel/user-space communication only (both directions)
    - vezzy-fnord 11y ago
      does that mean mach ipc provides typed interface registration / binding between userland processes? Yes, practically all Mach subsystems other than virtual memory have ports implicitly created for them. This includes tasks (address spaces) and threads (units of CPU time), which together form processes.
boardwaalk 11y ago
Practical question: If I was to get some cheap hardware, like a Chromebook, could I get a Debian GNU/Hurd development environment dual booting on it? What would I have available? I'd at least want a decent shell, tmux and vim and support for the display's full resolution. I suppose the easier route would be a virtual machine, but I think being immersed in it would be good. I've wanted to get into kernel or other low-level development for a while and this seems like a perfect entry point. Linux seems too complicating on the surface which is intimidating. And politics suck (see systemd).
- vezzy-fnord 11y ago
  bash, tmux and vim are all supported. You can even have a full Xfce desktop. Hardware support is unfortunately flaky, in part because the GNU Mach code lacks some features like PCI MSIs and uses drivers from the Linux 2.6 era through DDE. Virtualization is definitely the route through which people run it. The rump integration will hopefully change this situation, assuming it advances forward.
  - boardwaalk 11y ago
    I figured this would be about the size of things. I suppose I'll start with a VM. I realized I mentioned Chromebooks in my first post -- reminder to anyone that ARM is not supported, so get an Intel-based model.
Qwertious 11y ago
Linking to the slides without the actual presentation should be punishable by death. The entire point of slides is to supplement the presenter's speech, so if they're completely self-explanatory without any talking, then you're doing your presentation wrong.
chrismaeda 11y ago
I'm not drunk on wine at an OS research conference but I have to ask anyway: Aren't microkernels obsoleted by hypervisors?
- socceroos 11y ago
  Not really, no. A microkernel can be a self-reliant, real-time, event driven OS which can be embedded in any number of specific appliances with a specific skillset for a specific job. A hypervisor is generally used as a way to run another bloated OS on top of a microkernel. Even if you were to run with the idea of containers directly on top of a hypervisor, you're still looking at more layers of indirection that are meant to generalise the use of the entire stack. Microkernel = specific. Hypervisor = microkernel with specific purpose of running general stacks on top. That's how I would distinguish the two in this instance.

121 points by Tsiolkovsky 11y ago

Link to the talk: http://gensho.acc.umu.se/pub/debian-meetings/2015/debconf15/Debian_GNUHurd_status_update.webm http://gensho.acc.umu.se/pub/debian-meetings/2015/debconf15/... Other DebConf15 videos: http://ftp.acc.umu.se/pub/debian-meetings/2015/debconf15/ http://ftp.acc.umu.se/pub/debian-meetings/2015/debconf15/

jfaucett 11y ago

the last document linked to at the end is a really good read too - http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.37.9653&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.37...., Also I agree with him on the fdisk/mke2fs, I've often asked myself the exact same question. aparently, its not a so popular opinion though, since they're always tucked away in sbin...

viraptor 11y ago
Not all linux systems do that. Arch symlinks everything to /usr/bin, I believe.

mindcrime 11y ago

It's awesome to see Hurd continuing to progress. It doesn't matter that "Hurd isn't ready for mainstream use" or any of that stuff. Even if it never becomes "ready for mainstream use" it's valuable to have people experimenting and researching alternative approaches. This and Plan9 both still play important roles, and I expect that, at a minimum, developments coming out of one or both projects will continue to influence Linux and other "mainstream" OS's. Besides, it's just plain fun to play around with operating systems. I think I have an extra box laying around here idle that would be a good place to experiment with Hurd a bit.

vezzy-fnord 11y ago

The preliminary rump kernel support that arrived just a bit more than a week ago is quite promising, if still quite rudimentary. That said, a patched MPlayer linking to rump libraries has been able to play OGG files on a Hurd system: https://lists.gnu.org/archive/html/bug-hurd/2015-08/msg00027.html https://lists.gnu.org/archive/html/bug-hurd/2015-08/msg00027... Though obviously in the very initial stages, the implications of this development taken to its end could be enormous. Have the GNU Hurd implement translator or server interfaces to the rump drivers, plug it on top of an immutable package and configuration management suite like Guix (actually already being done), and for the first time in history you have a complete, general-purpose microkernel-based Unix-compatible (but beyond plain Unix) system that could rival the likes of GNU/Linux. The great thing about it is that the Hurd isn't afraid to break POSIX semantics where beneficial. This makes it have, among other things, processes running under multiple uids, true Plan 9-style namespacing, unprivileged mounts, token-based authentication, and so forth. I've been playing with it on-and-off (including cross-toolchains) and been pondering about Mach kernel emulation on Linux. A pure userland server would be most desirable, but faces real changes with modeling port rights as file descriptors sent over ancillary data, how to reliably handle user-level page fault handling on a per-application basis, mapping VM regions across address space boundaries and properly emulating RPC. Kip Macy did successful work on a port of OSF Mach from the MkLinux sources as a FreeBSD kernel module for use with launchd, XPC and other OS X modules, so I think that might end up being a concession I'll look into taking one day. I'm occupied with other projects in the meantime. It'd be also funny to have Linux "eat" itself in such a manner by becoming a host for the Hurd. Poetic justice, I suppose. But I'll wait to see if the rump kernel stuff pans out first. (MINIX 3 is great too, much better microkernel, but it's more meant for reliability rather than exposing end-user flexibility like Hurd does with translators. It also doesn't come close to the ~81% of Debian building as is with Hurd, and the MINIX devs themselves are positioning it as an embedded platform rather than general-purpose.)

nextos 11y ago
Guix is becoming increasingly ready for prime time. There's one big HPC install in Germany already running it. I think it's past the toy stage. I guess Guix + Hurd is a great setup, with innovative features in the kernel and userland. Both very pure in their respective ways, with Guix having functional package management, and Hurd keeping the kernel minimal so that crashes or security breaches are not catastrophic.

VLM 11y ago

The "top dumb issues" page for porters has value even for non-hurd users. In terms of "don't do this". Its on page 13.

jasonoliveira 11y ago

● Top dumb issues ● Not linux or BSD? #include <windows.h> can someone explain the above to me? the slides didn't make sense.

koenigdavidmj 11y ago
Poorly written preprocessor bits in C code that assume that anything anyone ever runs is either Linux, BSD (including OS X), or Windows. They basically write the following: #ifdef LINUX // include Linux headers #elif defined(BSD) // include BSD headers #else // include Windows headers. What else would it possibly be? #endif The "mach.h" one is similar: people make a bad assumption that OS X is the only Mach kernel people use. (Hurd is obviously a counterpoint, just an unpopular one.)
- ianremsen 11y ago
  I think it's wrong to call it poorly written because the programmer only accounted their C preprocessor code for ~99.99% of desktop and mobile operating systems. The diminishing return for avoiding this approach is so impalpable it hurts. Windows, OSX/iOS/BSD, and Linux/Android literally are everything anyone ever uses, within rounding error.
  - ori_b 11y ago
    The diminishing return approach is this: #ifdef LINUX // include Linux headers #elif defined(BSD) // include BSD headers #elif defined(WINDOWS) // include Windows headers. # else #error "unsupported system" #endif
  - viraptor 11y ago
    If you develop for different platforms then it may actually make sense to do a "if linux / elif bsd / elif windows / else #error unsupported". At least it gives you a proper answer in case you forgot you have some wild crosscompiler in your path. But I understand it's not a popular concern...

ai_ja_nai 11y ago

>Hardware support >● i686 >● start of 64bit support WTF? They started 30 years ago and still on an 32bit architecture nobody cares anymore? Beside that, I really don't get the point of all this, seriously. Not from a technological benefit, be warned, but from a user perspective the benefit is almost intangible. Why should the average sysadmin care? Linux is a rock solid kernel that does everything, what is the niche that Hurd is filling?

soong 11y ago
>>Beside that, I really don't get the point of all this, seriously. Not from a technological benefit, be warned, but from a user perspective the benefit is almost intangible. Why should the average sysadmin care? Linux is a rock solid kernel that does everything, what is the niche that Hurd is filling? Well, that's debatable. Linux just crossed the 20 MLOCs bar. Granted, this is mostly driver code which you will only need a small subset of, but still: Linux is a huge complex blob. Linus himself said it's bloated years ago[1] and that it has a high entry barrier for new developers due to its complexity[can't find the link right now]. My guess would be that this is partly due to its classic monolithic architecture and partly due to their developement model[2]. After working with Plan9 for a while I can't help but take my hat off to the beauty and simplicity of the system. Especially as a sysadmin your life would be so much easier. No LDAP, no Kerberos, no NFS, no need to update or deploy software on/to terminals. But as it stands we're living in a land where every computer thinks it is a Mainframe. Linux has done a great deal for OSS and will without question be around for quite some time but you have to wonder how computing would look like today, if history had taken a few diffenent turns. FWIW: Is it just me, or is OS R&D gaining a little more traction lately after it almost stagnated in the 2000s ? [1] http://www.cnet.com/news/linus-torvalds-linux-is-bloated/ http://www.cnet.com/news/linus-torvalds-linux-is-bloated/ [2] https://en.wikipedia.org/wiki/Criticism_of_Linux#Kernel_code_quality https://en.wikipedia.org/wiki/Criticism_of_Linux#Kernel_code...
- viraptor 11y ago
  I think there was some serious experimentation in 2000s, just not in the area of popular systems. Plan9 started to hit the news properly around then. A few OSes in managed languages were developed: MS Singularity, and smaller experiments like JNode and SharpOS.

viraptor 11y ago

I was hoping they will mention something about crosshurd, but there was nothing, and crosshurd doesn't work on recent debian anymore. I'd love to see some way to install from an existing system.

vezzy-fnord 11y ago
You can either follow my guide [1] based on updated cross-gnu scripts, or try out gnuxc [2] for bootstrapping a full Hurd distribution, with the caveat that it requires your host to be Fedora. [1] http://blog.darknedgy.net/technology/2015/07/25/0/ http://blog.darknedgy.net/technology/2015/07/25/0/ [2] https://github.com/dm0-/gnuxc https://github.com/dm0-/gnuxc
- viraptor 11y ago
  I found this bit in your post interesting: "kdbus (which was called “neutered Mach IPC” by Neal Walfield)" - does that mean mach ipc provides typed interface registration / binding between userland processes? I admit, I know very little about mach and thought it was for kernel/user-space communication only (both directions)
  - vezzy-fnord 11y ago
    does that mean mach ipc provides typed interface registration / binding between userland processes? Yes, practically all Mach subsystems other than virtual memory have ports implicitly created for them. This includes tasks (address spaces) and threads (units of CPU time), which together form processes.

boardwaalk 11y ago

Practical question: If I was to get some cheap hardware, like a Chromebook, could I get a Debian GNU/Hurd development environment dual booting on it? What would I have available? I'd at least want a decent shell, tmux and vim and support for the display's full resolution. I suppose the easier route would be a virtual machine, but I think being immersed in it would be good. I've wanted to get into kernel or other low-level development for a while and this seems like a perfect entry point. Linux seems too complicating on the surface which is intimidating. And politics suck (see systemd).

vezzy-fnord 11y ago
bash, tmux and vim are all supported. You can even have a full Xfce desktop. Hardware support is unfortunately flaky, in part because the GNU Mach code lacks some features like PCI MSIs and uses drivers from the Linux 2.6 era through DDE. Virtualization is definitely the route through which people run it. The rump integration will hopefully change this situation, assuming it advances forward.
- boardwaalk 11y ago
  I figured this would be about the size of things. I suppose I'll start with a VM. I realized I mentioned Chromebooks in my first post -- reminder to anyone that ARM is not supported, so get an Intel-based model.

Qwertious 11y ago

Linking to the slides without the actual presentation should be punishable by death. The entire point of slides is to supplement the presenter's speech, so if they're completely self-explanatory without any talking, then you're doing your presentation wrong.

chrismaeda 11y ago

I'm not drunk on wine at an OS research conference but I have to ask anyway: Aren't microkernels obsoleted by hypervisors?

socceroos 11y ago
Not really, no. A microkernel can be a self-reliant, real-time, event driven OS which can be embedded in any number of specific appliances with a specific skillset for a specific job. A hypervisor is generally used as a way to run another bloated OS on top of a microkernel. Even if you were to run with the idea of containers directly on top of a hypervisor, you're still looking at more layers of indirection that are meant to generalise the use of the entire stack. Microkernel = specific. Hypervisor = microkernel with specific purpose of running general stacks on top. That's how I would distinguish the two in this instance.