18 ms·

Kind of weird to see linux called out as too big, and then the ack that writing a ton of device drivers is hard and will take time. Isn't that a large part of

by taeric 3y ago

Kind of weird to see linux called out as too big, and then the ack that writing a ton of device drivers is hard and will take time. Isn't that a large part of the source tree? Realizing I haven't even tried compiling a kernel in a long long time. Feels oddly sad to say.

weberer 3y ago
Yes, its mostly drivers. Here's a breakdown of the Linux kernel by lines of code: https://upload.wikimedia.org/wikipedia/commons/f/f5/Sankey_Diagram_of_Linux_Kernel_Source_Lines_of_Code.jpg https://upload.wikimedia.org/wikipedia/commons/f/f5/Sankey_D...
- bee_rider 3y ago
  That’s a neat way of visualizing it. It is interesting that the “remainder” chunk of the drivers is so big. Wonder if they need some more categories.
  - stavros 3y ago
    I was expecting a block chart, kind of disappointed it isn't.
- winter_blue 3y ago
  Wow, I'm surprised by how small the file systems are. On the smaller side, Ext4 at just 21,515 lines, and on the larger side, Btrfs at 55,758 lines. It almost makes me think that writing a new file system in Rust, might actually be a tractable project.
  - PaulDavisThe1st 3y ago
    [flagged]
    - meepmorp 3y ago
      It's a good thing that you warned them off, or they might've accidentally gone and done something interesting or enjoyable.
      - PaulDavisThe1st 3y ago
        Yep, because everyone who is going to do something interesting or enjoyable starts with the question "should I use language X because it's cool?". Look, I'm as into cool FS-hacks as anyone, but writing a new filesystem has 99+ problems and the language used is either the last of them, or it isn't even in the set. Granted, you could (on Linux) write a little pseudo-filesystem (e.g. /proc or /sys) and not face any of the issues involved in extN or btrfs etc. I didn't get the sense that this was what the OP intended, but I could be wrong.
        Paul-Craft 3y ago
        Writing a basic filesystem using FUSE is pretty simple, fun, and can occasionally be useful.
      - serf 3y ago
        I think it's a great project to tackle. Here are my criticisms : 1) LoC doesn't indicate complexity. When you're looking at a code base and you see that it's 65000 lines, that by no means should be an indicator of whether or not a re-write is easily feasible -- especially with a filesystem. 2) Language shouldn't be a motivator for a re-write without considering why the language offers advantages, the end users don't care about which file extension is littered across the source tree. Both of those tendencies (one to re-write everything in your favorite language, and two to come across with an idea that you understand how complex a mechanism is simply from LoC) indicate to me, anecdotally of course, a certain lack of experience; and I have a hard time imagining someone with those concerns tackling a re-write of a filesystem that gains 'the audience' any benefits. If it's just for the sake of personal toys and fooling around with your own machine I feel as if those motivations are fine, but I have concerns if 'a new filesystem in Rust' comes from what I view as shaky initial motivations.
  - Nereuxofficial 3y ago
    I also thought that'd be a fun project. BTRFS seems to have the largest feature set although many features are experimental but maybe there would be a place for something smaller that scales well and maybe has some of the new features like deduplication
  - lutoma 3y ago
    I'm writing a hobby kernel/operating system and one of the things that surprised me was how simple ext2 is. It really isn't much different from fat32 in many ways, just a bit more flexible and designed in a more future-proof way. Obviously ext2 is a very outdated filesystem by today's standards, but considering ext3 and ext4 are essentially just some extensions slapped onto ext2, I still didn't expect it. Another thing I found really impressive was just how resilient e2fsck is. During early development of my ext2 code I frequently mangled the file system in a myriad of weird ways, but no matter how much I broke things, e2fsck was almost always able to restore the file system into a reasonable state. (For the curious, there is some excellent documentation on the ext2 file system at https://www.nongnu.org/ext2-doc/ext2.html https://www.nongnu.org/ext2-doc/ext2.html that I mostly used for my implementation of it. Sadly no such thing seems to really exist for ext3/ext4).
  - JohnFen 3y ago
    Filesystems tend not to be all that complex to implement.
    - effie 3y ago
      Simple filesystems - those oriented at speed, without integrity checking, compression, encryption and raid-like features. Which are extremely useful and desirable in a filesystem. When people tried to do a full-featured filesystem, with a remarkable exception (ZFS) it's either a disappointing unfinished mess (btrfs) or a promising, but a long-term project taking years (bcachefs). Making a filesystem with strong support of important features is an extremely demanding project.
      - midoridensha 3y ago
        Don't forget: buggy filesystem code can easily lead to catastrophic data loss, which ruins your filesystem's reputation. Lines of code isn't the only important metric. A project with a smaller number of LoC can take a long time to fully test and debug and validate.
      - Foobar8568 3y ago
        Hammer2 comes to mind as well..
        effie 3y ago
        Does it run on Linux or ordinary BSD? Does any company use it?
      - spookie 3y ago
        I'm sorry, as I keep hearing these criticisms around BTRFS, I still don't know why. What is so messy about it? Honest question
        janneke 3y ago
        Have you tried reading the code? It's twice as large (107KLOC) as ext4 (45KLOC), where ext4 is viewed as a natural evolution of ext2 (7KLOC). 45KLOC of C code already sounds really scary to me, although yeah, journaling is really nice to have.
  - rcxdude 3y ago
    The difficulty in filesystems is generally getting to the point of high enough reliability that people trust then with their data. This is why development and adoption of new ones tends to be slow, not the quantity or complexity of implementation. So for a hobby it's quite accessible because that expectation of reliability isn't there.
- broodbucket 3y ago
  Wow arch/arm (which is 32bit) is bigger than x86 and powerpc which have 32/64 under the same subsystem. I guess it's mostly device tree bindings?
- askiiart 3y ago
  For anyone wondering, drivers are 5205102 out of 7616785 lines, or about 68.34% of all lines.
cbsmith 3y ago
With a microkernel model, most of those drivers would run in user space, not kernel space. So it's possible for the device drivers to be large but the GNU HURD kernel itself to be much smaller than the Linux kernel.
molticrystal 3y ago
LoC is a metric for manageability of the code base and compile time, and you might need other metrics to judge those two factors. For real world use wheen comparing OS you would generally go for other metrics like resource usage, multi threading bottlenecks, throughput of network packet and their processing, context switches(or equivalent) per second, and dozens of other things that would be paragraphs in length before you get to LoC.
thrillgore 3y ago
I've been playing with Redox a lot lately. By comparison, everything is big compared to Redox.
_8j50 3y ago
Hurd is a microkernel so it is lean by design. I suspect that statement was a dig on perceived architectural flaw of Linux.
- taeric 3y ago
  I get that. But it is still an unfair dig when a large chunk of the code that is being pointed at is the code that they will have to write? (Granted, they can put an easier line in the sand on how far back they go on what they support hardware wise, but still...)
  - pengaru 3y ago
    It's not about the code existing, it's about the code being an integral part of a monolithic architecture vs. "servers" facilitating a microkernel design like hurd/mach.
    - taeric 3y ago
      But that is silly? If I have to audit the code for my system running, I would have to audit the code for each of the "servers" as well? If not, why not? To that end, how different is it, actually? And what are the other tradeoffs?
      - pengaru 3y ago
        The servers are run in isolated address spaces like processes, taking advantage of MMU protection for more of the low-level facilities. Ask yourself why it's safer to use FUSE filesystems on Linux when mounting untrusted block devices/image files than the in-kernel filesystem drivers and maybe it'll become clear why this is advantageous. Or if you're familiar with containers and why that's a good thing, you can think of it as containerizing the kernel's subsystems/drivers.
        taeric 3y ago
        This... didn't answer my question? For many that are concerned with containers and such, they also have to audit the container process code. Such that, fine, try to use that to inflate the numbers on Linux. Pointing at literally decades of device drivers as why the size of the code is bad, while acknowledging that it will take time to write device drivers so this will work on a respectable amount of hardware just feels weird. That all said, please don't take this as a criticism of any of these ideas. I confess seeing Hurd in the headlines gave me a smile. Is nice to see people can still work on many ideas out there. I am only questioning the "size of code bad!" right next to "it will take us a long time to replicate a large portion of the code we just pointed at."
dwheeler 3y ago
I agree, it's odd. I don't have anything recent, but back in 2004, the majority of the Linux kernel code was in its drivers: https://dwheeler.com/essays/linux-kernel-cost.html https://dwheeler.com/essays/linux-kernel-cost.html I expect that most of the current Linux kernel code is also for handling hardware (that is, drivers + the code to handle various architectures).
- emmericp 3y ago
  66% was drivers in 2019.
- AdmiralAsshat 3y ago
  Could one not make the case that maybe device driver support doesn't belong in the kernel itself?
  - boricj 3y ago
    NetBSD has architectured their drivers to be basically static archives reusable anywhere. It's part of the NetBSD rump kernel.
    - rjsw 3y ago
      The drivers are not used in that way in NetBSD, they are either compiled into the kernel or built as modules just like in Linux, the graphics drivers even use the source code from Linux.
    - einpoklum 3y ago
      What do you mean by "reusable everywhere"? I mean, where could they be used other than in a NetBSD kernel?
      - boricj 3y ago
        Anywhere. There are lots of examples on the topic ; running NetBSD driver code inside NetBSD's user-land is the simplest use-case (https://www.netbsd.org/docs/rump/sptut.html https://www.netbsd.org/docs/rump/sptut.html), but that code is portable without modifications to just about any context possible. Check out Antti Kantee's The Design and Implementation of the Anykernel and Rump Kernels if you want details on the architecture.
jasoneckert 3y ago
And given what the modern Linux kernel does, alongside the power of modern systems (even embedded ones), it's also difficult to argue that the Linux kernel is too large by any stretch of the imagination.
- genewitch 3y ago
  my entire kernel, including all drivers needed for my system except the nvidia blob is 11MB - 10876160 bytes. i don't optimize for size, necessarily, i'm sure i could get it down a lot more if i disabled most of the "default enabled" stuff in a modern kernel that assures it will boot on a majority of systems.
muixoozie 3y ago
Can get a sense of how long it takes to compile here https://openbenchmarking.org/test/pts/build-linux-kernel https://openbenchmarking.org/test/pts/build-linux-kernel surprisingly fast with modern processors.
- genewitch 3y ago
  I used to compile kernels on a 2012 HP quad-CPU, 80 thread machine. It would finish in a minute and a half or so. I got a ryzen 5850 to replace it, it also takes a minute and a half or so - but uses 1/4th the power at the wall. 32 threads vs 80, 250W vs 1000+W
- taeric 3y ago
  My memory was spending as much time learning on the different options as anything else. Though, I think most of my memory was all of the config for FreeBSD. Compiling Firefox was the beast that I remember. Only thing I still compile is Emacs, and that is silly fast on modern computers. Really, the speed of compilation is such that I understand why so many source distribution methods are as popular as they are now. Feels funny to have come to that point so effectively.
  - anonymousiam 3y ago
    The big break (for me) in configuring the options was the very welcome addition of "make menuconfig" to replace the original "make config". As the number of supported filesystems/drivers/options increased, it could take more than an hour to get through a "make config". I haven't built my own kernel for more than 15 years, but I probably built more than a dozen before that, each of them re-compiled upon every new release (going back to 0.98).
- stusmall 3y ago
  I love how much things have progressed. I remember I used to have to set up a clean build to run while I was at school so it could be done when I got back. Now it is done within seconds.
  - 3y ago
gattilorenz 3y ago
Heh, not a particulary in-depth article in any way, to be honest. "the GNU/Hurd is Unix(POSIX)-compatible, so most things work"... well that's to be said for MacOS, QNX, etc. In practice, where is the "65% of the Debian archive can be built for the Hurd" figure coming from? I'm sure it can be built in theory, how much does it require source modifications?
- rekado 3y ago
  Here's a list of failing packages with reasons for failure: https://people.debian.org/~sthibault/failed_packages.txt https://people.debian.org/~sthibault/failed_packages.txt And here's the overview on how many packages have successfully been built vs how many have failed: https://buildd.debian.org/status/architecture.php?a=hurd-i386&suite=sid https://buildd.debian.org/status/architecture.php?a=hurd-i38... The number of packages that can be built have been built; it is not a theoretical number.
  - gattilorenz 3y ago
    thanks for the pointer, it's actually quite interesting. Apparently even (most of) X11 should work then?
    - rekado 3y ago
      Yes, X11 has been working for a long time. Here's a screenshot from 2013: https://lwn.net/Articles/555566/ https://lwn.net/Articles/555566/
darksaints 3y ago
I understand what you're saying, but both are true. 1. Linux is too big. Even if we exclude the drivers, which are a majority of the kernel codebase, linux is still a massive kernel. Most microkernels are small enough to fit in the L3 cache, some are small enough to fit in the L2 cache. Linux, even if we could exclude the drivers, doesn't even come close. This inability to fully cache the kernel ends up negating the primary benefit that monolithic kernels have, which is performance. Linux spends more time bouncing around the cache than a modern microkernel takes to do an extra syscall. To add to this, unless you are compiling your kernel specifically for your machine, your kernel is going to be a bloated compromise of the set of drivers that your system is most likely to see. That means that you will have dozens, if not hundreds of drivers compiled into your kernel which will never need to be used. It's also too big in the sense that there are now far too many lines of code to be able to effectively monitor for vulnerabilites. That codebase size won't change when having a microkernel with the same number of drivers, but what will change is the ability of a single vulnerability to compromise your whole system. You have effectively compartmentalized vulnerabilities to their own address space. So in that sense, big codebases are a huge problem for monolithic kernels, but only a small problem for microkernels. 2. It will take time to write a ton of device drivers, simply because most of them aren't written yet. To that extent, Linux most definitely has an incumbent advantage. But that isn't to say that microkernels also have an advantage here. Anyone who has written a driver knows how much the ability to iterate quickly is helpful. Drivers in userspace can be written directly on the system that they will run on, with no virtualization or kernel reboots necessary, and with incredibly fast start/restart/stop. Userspace drivers are dramatically simpler to write, simpler to build, simpler to monitor, simpler to test, and simpler to distribute. With all that being said, I definitely don't think Hurd is the future. But what could change the world is already out there but not really ready to use at the moment. One possibility would be an open-sourcing of QNX which is an amazing and incredibly mature microkernel OS, possibly the most mature in the world. If there are any bored billionaires out there reading this, please buy QNX from Blackberry and do that. Another possibility would be to put a shit ton of effort behind the SeL4 userspace, which is just too damn hard to do anything practical with at the moment.
- bityard 3y ago
  > Most microkernels are small enough to fit in the L3 cache Is that important, though? As soon as you start processing any meaningful amount of data, won't your OS be evicted from the cache anyway? > unless you are compiling your kernel specifically for your machine, your kernel is going to be a bloated compromise of the set of drivers that your system is most likely to see. That means that you will have dozens, if not hundreds of drivers compiled into your kernel which will never need to be used. I'm not sure this is true. Device drivers are typically compiled as modules, which are only loaded into the kernel if needed. Common wisdom among Linux power users for the last decade or so has been that compiling your own kernel for any tangible performance/memory improvement is basically futile. Are there exceptions to this that you are aware of, and can elaborate on?
marcosdumay 3y ago
> Isn't that a large part of the source tree? For Linux, yes. For a microkernel, those are different trees of userspace software. Not to say, Hurd itself is userspace software, that runs over a microkernel (included on the distribution).
- uxp8u61q 3y ago
  > For Linux, yes. For a microkernel, those are different trees of userspace software. If that were the argument, then the comparison with Linux would be meaningless. Hurd is much bigger than freertos, should everyone switch?

by taeric 3y ago

weberer 3y ago

Yes, its mostly drivers. Here's a breakdown of the Linux kernel by lines of code: https://upload.wikimedia.org/wikipedia/commons/f/f5/Sankey_Diagram_of_Linux_Kernel_Source_Lines_of_Code.jpg https://upload.wikimedia.org/wikipedia/commons/f/f5/Sankey_D...

bee_rider 3y ago
That’s a neat way of visualizing it. It is interesting that the “remainder” chunk of the drivers is so big. Wonder if they need some more categories.
- stavros 3y ago
  I was expecting a block chart, kind of disappointed it isn't.
winter_blue 3y ago
Wow, I'm surprised by how small the file systems are. On the smaller side, Ext4 at just 21,515 lines, and on the larger side, Btrfs at 55,758 lines. It almost makes me think that writing a new file system in Rust, might actually be a tractable project.
- PaulDavisThe1st 3y ago
  [flagged]
  - meepmorp 3y ago
    It's a good thing that you warned them off, or they might've accidentally gone and done something interesting or enjoyable.
    - PaulDavisThe1st 3y ago
      Yep, because everyone who is going to do something interesting or enjoyable starts with the question "should I use language X because it's cool?". Look, I'm as into cool FS-hacks as anyone, but writing a new filesystem has 99+ problems and the language used is either the last of them, or it isn't even in the set. Granted, you could (on Linux) write a little pseudo-filesystem (e.g. /proc or /sys) and not face any of the issues involved in extN or btrfs etc. I didn't get the sense that this was what the OP intended, but I could be wrong.
      - Paul-Craft 3y ago
        Writing a basic filesystem using FUSE is pretty simple, fun, and can occasionally be useful.
    - serf 3y ago
      I think it's a great project to tackle. Here are my criticisms : 1) LoC doesn't indicate complexity. When you're looking at a code base and you see that it's 65000 lines, that by no means should be an indicator of whether or not a re-write is easily feasible -- especially with a filesystem. 2) Language shouldn't be a motivator for a re-write without considering why the language offers advantages, the end users don't care about which file extension is littered across the source tree. Both of those tendencies (one to re-write everything in your favorite language, and two to come across with an idea that you understand how complex a mechanism is simply from LoC) indicate to me, anecdotally of course, a certain lack of experience; and I have a hard time imagining someone with those concerns tackling a re-write of a filesystem that gains 'the audience' any benefits. If it's just for the sake of personal toys and fooling around with your own machine I feel as if those motivations are fine, but I have concerns if 'a new filesystem in Rust' comes from what I view as shaky initial motivations.
- Nereuxofficial 3y ago
  I also thought that'd be a fun project. BTRFS seems to have the largest feature set although many features are experimental but maybe there would be a place for something smaller that scales well and maybe has some of the new features like deduplication
- lutoma 3y ago
  I'm writing a hobby kernel/operating system and one of the things that surprised me was how simple ext2 is. It really isn't much different from fat32 in many ways, just a bit more flexible and designed in a more future-proof way. Obviously ext2 is a very outdated filesystem by today's standards, but considering ext3 and ext4 are essentially just some extensions slapped onto ext2, I still didn't expect it. Another thing I found really impressive was just how resilient e2fsck is. During early development of my ext2 code I frequently mangled the file system in a myriad of weird ways, but no matter how much I broke things, e2fsck was almost always able to restore the file system into a reasonable state. (For the curious, there is some excellent documentation on the ext2 file system at https://www.nongnu.org/ext2-doc/ext2.html https://www.nongnu.org/ext2-doc/ext2.html that I mostly used for my implementation of it. Sadly no such thing seems to really exist for ext3/ext4).
- JohnFen 3y ago
  Filesystems tend not to be all that complex to implement.
  - effie 3y ago
    Simple filesystems - those oriented at speed, without integrity checking, compression, encryption and raid-like features. Which are extremely useful and desirable in a filesystem. When people tried to do a full-featured filesystem, with a remarkable exception (ZFS) it's either a disappointing unfinished mess (btrfs) or a promising, but a long-term project taking years (bcachefs). Making a filesystem with strong support of important features is an extremely demanding project.
    - midoridensha 3y ago
      Don't forget: buggy filesystem code can easily lead to catastrophic data loss, which ruins your filesystem's reputation. Lines of code isn't the only important metric. A project with a smaller number of LoC can take a long time to fully test and debug and validate.
    - Foobar8568 3y ago
      Hammer2 comes to mind as well..
      - effie 3y ago
        Does it run on Linux or ordinary BSD? Does any company use it?
    - spookie 3y ago
      I'm sorry, as I keep hearing these criticisms around BTRFS, I still don't know why. What is so messy about it? Honest question
      - janneke 3y ago
        Have you tried reading the code? It's twice as large (107KLOC) as ext4 (45KLOC), where ext4 is viewed as a natural evolution of ext2 (7KLOC). 45KLOC of C code already sounds really scary to me, although yeah, journaling is really nice to have.
- rcxdude 3y ago
  The difficulty in filesystems is generally getting to the point of high enough reliability that people trust then with their data. This is why development and adoption of new ones tends to be slow, not the quantity or complexity of implementation. So for a hobby it's quite accessible because that expectation of reliability isn't there.
broodbucket 3y ago
Wow arch/arm (which is 32bit) is bigger than x86 and powerpc which have 32/64 under the same subsystem. I guess it's mostly device tree bindings?
askiiart 3y ago
For anyone wondering, drivers are 5205102 out of 7616785 lines, or about 68.34% of all lines.

cbsmith 3y ago

With a microkernel model, most of those drivers would run in user space, not kernel space. So it's possible for the device drivers to be large but the GNU HURD kernel itself to be much smaller than the Linux kernel.

molticrystal 3y ago

LoC is a metric for manageability of the code base and compile time, and you might need other metrics to judge those two factors. For real world use wheen comparing OS you would generally go for other metrics like resource usage, multi threading bottlenecks, throughput of network packet and their processing, context switches(or equivalent) per second, and dozens of other things that would be paragraphs in length before you get to LoC.

thrillgore 3y ago

I've been playing with Redox a lot lately. By comparison, everything is big compared to Redox.

_8j50 3y ago

Hurd is a microkernel so it is lean by design. I suspect that statement was a dig on perceived architectural flaw of Linux.

taeric 3y ago
I get that. But it is still an unfair dig when a large chunk of the code that is being pointed at is the code that they will have to write? (Granted, they can put an easier line in the sand on how far back they go on what they support hardware wise, but still...)
- pengaru 3y ago
  It's not about the code existing, it's about the code being an integral part of a monolithic architecture vs. "servers" facilitating a microkernel design like hurd/mach.
  - taeric 3y ago
    But that is silly? If I have to audit the code for my system running, I would have to audit the code for each of the "servers" as well? If not, why not? To that end, how different is it, actually? And what are the other tradeoffs?
    - pengaru 3y ago
      The servers are run in isolated address spaces like processes, taking advantage of MMU protection for more of the low-level facilities. Ask yourself why it's safer to use FUSE filesystems on Linux when mounting untrusted block devices/image files than the in-kernel filesystem drivers and maybe it'll become clear why this is advantageous. Or if you're familiar with containers and why that's a good thing, you can think of it as containerizing the kernel's subsystems/drivers.
      - taeric 3y ago
        This... didn't answer my question? For many that are concerned with containers and such, they also have to audit the container process code. Such that, fine, try to use that to inflate the numbers on Linux. Pointing at literally decades of device drivers as why the size of the code is bad, while acknowledging that it will take time to write device drivers so this will work on a respectable amount of hardware just feels weird. That all said, please don't take this as a criticism of any of these ideas. I confess seeing Hurd in the headlines gave me a smile. Is nice to see people can still work on many ideas out there. I am only questioning the "size of code bad!" right next to "it will take us a long time to replicate a large portion of the code we just pointed at."

dwheeler 3y ago

I agree, it's odd. I don't have anything recent, but back in 2004, the majority of the Linux kernel code was in its drivers: https://dwheeler.com/essays/linux-kernel-cost.html https://dwheeler.com/essays/linux-kernel-cost.html I expect that most of the current Linux kernel code is also for handling hardware (that is, drivers + the code to handle various architectures).

emmericp 3y ago
66% was drivers in 2019.
AdmiralAsshat 3y ago
Could one not make the case that maybe device driver support doesn't belong in the kernel itself?
- boricj 3y ago
  NetBSD has architectured their drivers to be basically static archives reusable anywhere. It's part of the NetBSD rump kernel.
  - rjsw 3y ago
    The drivers are not used in that way in NetBSD, they are either compiled into the kernel or built as modules just like in Linux, the graphics drivers even use the source code from Linux.
  - einpoklum 3y ago
    What do you mean by "reusable everywhere"? I mean, where could they be used other than in a NetBSD kernel?
    - boricj 3y ago
      Anywhere. There are lots of examples on the topic ; running NetBSD driver code inside NetBSD's user-land is the simplest use-case (https://www.netbsd.org/docs/rump/sptut.html https://www.netbsd.org/docs/rump/sptut.html), but that code is portable without modifications to just about any context possible. Check out Antti Kantee's The Design and Implementation of the Anykernel and Rump Kernels if you want details on the architecture.

jasoneckert 3y ago

And given what the modern Linux kernel does, alongside the power of modern systems (even embedded ones), it's also difficult to argue that the Linux kernel is too large by any stretch of the imagination.

genewitch 3y ago
my entire kernel, including all drivers needed for my system except the nvidia blob is 11MB - 10876160 bytes. i don't optimize for size, necessarily, i'm sure i could get it down a lot more if i disabled most of the "default enabled" stuff in a modern kernel that assures it will boot on a majority of systems.

muixoozie 3y ago

Can get a sense of how long it takes to compile here https://openbenchmarking.org/test/pts/build-linux-kernel https://openbenchmarking.org/test/pts/build-linux-kernel surprisingly fast with modern processors.

genewitch 3y ago
I used to compile kernels on a 2012 HP quad-CPU, 80 thread machine. It would finish in a minute and a half or so. I got a ryzen 5850 to replace it, it also takes a minute and a half or so - but uses 1/4th the power at the wall. 32 threads vs 80, 250W vs 1000+W
taeric 3y ago
My memory was spending as much time learning on the different options as anything else. Though, I think most of my memory was all of the config for FreeBSD. Compiling Firefox was the beast that I remember. Only thing I still compile is Emacs, and that is silly fast on modern computers. Really, the speed of compilation is such that I understand why so many source distribution methods are as popular as they are now. Feels funny to have come to that point so effectively.
- anonymousiam 3y ago
  The big break (for me) in configuring the options was the very welcome addition of "make menuconfig" to replace the original "make config". As the number of supported filesystems/drivers/options increased, it could take more than an hour to get through a "make config". I haven't built my own kernel for more than 15 years, but I probably built more than a dozen before that, each of them re-compiled upon every new release (going back to 0.98).
stusmall 3y ago
I love how much things have progressed. I remember I used to have to set up a clean build to run while I was at school so it could be done when I got back. Now it is done within seconds.
- 3y ago

gattilorenz 3y ago

Heh, not a particulary in-depth article in any way, to be honest. "the GNU/Hurd is Unix(POSIX)-compatible, so most things work"... well that's to be said for MacOS, QNX, etc. In practice, where is the "65% of the Debian archive can be built for the Hurd" figure coming from? I'm sure it can be built in theory, how much does it require source modifications?

rekado 3y ago
Here's a list of failing packages with reasons for failure: https://people.debian.org/~sthibault/failed_packages.txt https://people.debian.org/~sthibault/failed_packages.txt And here's the overview on how many packages have successfully been built vs how many have failed: https://buildd.debian.org/status/architecture.php?a=hurd-i386&suite=sid https://buildd.debian.org/status/architecture.php?a=hurd-i38... The number of packages that can be built have been built; it is not a theoretical number.
- gattilorenz 3y ago
  thanks for the pointer, it's actually quite interesting. Apparently even (most of) X11 should work then?
  - rekado 3y ago
    Yes, X11 has been working for a long time. Here's a screenshot from 2013: https://lwn.net/Articles/555566/ https://lwn.net/Articles/555566/

darksaints 3y ago

I understand what you're saying, but both are true. 1. Linux is too big. Even if we exclude the drivers, which are a majority of the kernel codebase, linux is still a massive kernel. Most microkernels are small enough to fit in the L3 cache, some are small enough to fit in the L2 cache. Linux, even if we could exclude the drivers, doesn't even come close. This inability to fully cache the kernel ends up negating the primary benefit that monolithic kernels have, which is performance. Linux spends more time bouncing around the cache than a modern microkernel takes to do an extra syscall. To add to this, unless you are compiling your kernel specifically for your machine, your kernel is going to be a bloated compromise of the set of drivers that your system is most likely to see. That means that you will have dozens, if not hundreds of drivers compiled into your kernel which will never need to be used. It's also too big in the sense that there are now far too many lines of code to be able to effectively monitor for vulnerabilites. That codebase size won't change when having a microkernel with the same number of drivers, but what will change is the ability of a single vulnerability to compromise your whole system. You have effectively compartmentalized vulnerabilities to their own address space. So in that sense, big codebases are a huge problem for monolithic kernels, but only a small problem for microkernels. 2. It will take time to write a ton of device drivers, simply because most of them aren't written yet. To that extent, Linux most definitely has an incumbent advantage. But that isn't to say that microkernels also have an advantage here. Anyone who has written a driver knows how much the ability to iterate quickly is helpful. Drivers in userspace can be written directly on the system that they will run on, with no virtualization or kernel reboots necessary, and with incredibly fast start/restart/stop. Userspace drivers are dramatically simpler to write, simpler to build, simpler to monitor, simpler to test, and simpler to distribute. With all that being said, I definitely don't think Hurd is the future. But what could change the world is already out there but not really ready to use at the moment. One possibility would be an open-sourcing of QNX which is an amazing and incredibly mature microkernel OS, possibly the most mature in the world. If there are any bored billionaires out there reading this, please buy QNX from Blackberry and do that. Another possibility would be to put a shit ton of effort behind the SeL4 userspace, which is just too damn hard to do anything practical with at the moment.

bityard 3y ago
> Most microkernels are small enough to fit in the L3 cache Is that important, though? As soon as you start processing any meaningful amount of data, won't your OS be evicted from the cache anyway? > unless you are compiling your kernel specifically for your machine, your kernel is going to be a bloated compromise of the set of drivers that your system is most likely to see. That means that you will have dozens, if not hundreds of drivers compiled into your kernel which will never need to be used. I'm not sure this is true. Device drivers are typically compiled as modules, which are only loaded into the kernel if needed. Common wisdom among Linux power users for the last decade or so has been that compiling your own kernel for any tangible performance/memory improvement is basically futile. Are there exceptions to this that you are aware of, and can elaborate on?

marcosdumay 3y ago

> Isn't that a large part of the source tree? For Linux, yes. For a microkernel, those are different trees of userspace software. Not to say, Hurd itself is userspace software, that runs over a microkernel (included on the distribution).

uxp8u61q 3y ago
> For Linux, yes. For a microkernel, those are different trees of userspace software. If that were the argument, then the comparison with Linux would be meaningless. Hurd is much bigger than freertos, should everyone switch?