5 ms·

What the article is suggesting is that the Linux architecture wasn't affected by organisational pressures that closed-source systems face. That is to say that

by scribu 9y ago

What the article is suggesting is that the Linux architecture wasn't affected by organisational pressures that closed-source systems face. That is to say that subsytems were defined solely based on technical considerations, which is how it should be if the goal is sound engineering. Not sure what to make of the ratio between "specialists" and "generalists". A comparison to ratios from other projects would provide some helpful context.

nwmcsween 9y ago
Not organizational pressures so much but personal politics moreso as there probably quite a few subpar solutions in the kernel because of egos, etc.
- taeric 9y ago
  Isn't this exactly like in corporations?
  - dredmorbius 9y ago
    Interpersonal relations are probably more fluid and less persistant than organisational ones. Organisations are a superset of people, and demonstrate their own emergent properties, most notably organisational persistence beyond the founding remit. People tend not to outlive their founding remit. There's also the (general) case that kernel development doesn't create the self-propogating and self-funding structures you'll find within commercial (or even not-for-profit) organisations. All of which suggests a possibly looser binding strength. The possibility of an alternative development effort tackling a similar problem, under a different group and structure, and being considered not on profitability but on fitness to task likely also helps (modulo inefficiencies in assessing that fitness).
jacquesm 9y ago
> That is to say that subsy[s]tems were defined solely based on technical considerations, which is how it should be if the goal is sound engineering. That never was the goal and if it was it hasn't been achieved. Linux is like democracy, it isn't perfect but it is the best that we've got. Unlike democracy however it is fairly easy to get rid of Linux and replace it with something better.
- jacques_chester 9y ago
  > Unlike democracy however it is fairly easy to get rid of Linux and replace it with something better. Is it?
  - inopinatus 9y ago
    My first thought was "sure, FreeBSD". My second was "hmm, not always so easy".
  - jacquesm 9y ago
    Yes. Ironically, the hardest parts would be closed source drivers (Nvidia, several others). Linux after all is only a kernel. One interesting way in which you could do this is to build a wrapper around device drivers so that each device driver becomes a process on top of a microkernel. The userland can mostly remain unchanged as long as you expose the expected interfaces.
    - ifdefdebug 9y ago
      "only a kernel" Come on. You think that is the easy part? Just because everybody can hack up some "kernel" and boot their machine into it? I think you know the relevant Linus Torvalds quote in this context ...
      - jacquesm 9y ago
        I bootstrapped a QnX compatible kernel (with GNU userland) in about 2 years that was then used to power a bunch of special purpose routers. Yes, that's the easy part. As Jacques_Chester already remarked elsewhere in this thread the community part (herding cats) and getting the required buy-in is the hard part. The first can be done, I'm 100% sure of that, the second I'm not so sure. I'm kind of surprised that on HN of all places something like a kernel would be considered somehow beyond re-creation or requiring special god-like powers.
        ifdefdebug 9y ago
xyzzy_plugh 9y ago
That sounds nice, but realistically a ton of driver code is shoehorned in by sloppy corporate sponsors whose contributors stumble down the narrow, winding line between getting paid and making Linus happy.
lomnakkus 9y ago
> That is to say that subsytems were defined solely based on technical considerations, which is how it should be if the goal is sound engineering. I think that's too idealistic. As another sibling poster pointed out, it's more like a democracy... with all the attendant upsides and downsides. (Also, I'm sure there's a lot of interpersonal politics involved, even if it's all over email.) Don't get me wrong, the Linux kernel works surprisingly well and I rely on it for almost everything (including my livelihood), but if you really look at it, some of the subsystems are shockingly bad. I think a good example is containers/namespaces which have a ridiculously bad security record. (See "user namespaces".) Again, I'm sure the people working on these things had the best of intentions, are very competent generally, and were hampered by the "never break userspace" rule. However, if containers/namespaces were truly designed, it could have been done a lot better. (See "Zones" on Solaris et al.)
- X86BSD 9y ago
  I agree with you. But I think you're being generous when saying some of the subsystems and architecture are shockingly bad. Some of them are not even designed. If you look at them it's like "This isn't even useable!" The problem with namespaces and all the "container" tooling built on top of it is, as you noted, security. They were never designed to be secure from the start. It was not meant to be. And it was always puzzling to me why someone would use them to build containers on. Knowing it wasn't secure. As you also note Solaris got it right with zones, FreeBSD got it right with jails, there was plenty of options to use and build on. But as is the typical Linux story they had to do things there own way, and poorly. I still have no idea what Linux land is going to do for a filesystem. Since it's pretty clear ZFS is kept at arms length. Btrfs seems like the horse they are hitching to? Filesystems are hard. Which is why they can't seem to write a solid, reliable one and keep starting over. I think it's a big issue for them and is only going to get worse as data continues to explode exponentially. I wish zfs had been BSD licensed. And I wish Linux had simply adopted it as well. But those ships have sailed. Where to now?
  - lomnakkus 9y ago
    Agreed, I probably was being a bit too generous, but I usually try to be as "diplomatic" as possible :). I don't know much about the BSDs, but my impression was that jails don't really offer nearly enough to rival Zones. Zones is really "containers done right"[1]. Definitely agree on ZFS. I'm currently using ZoL on my Ubuntu servers, but I'm not going to try using it on anything else. (FWIW, it's working marvellously so far.) Personally, my current best hope for a good Linux FS at the moment is Bcachefs[2]. It's based on the proven Bcache. It's mostly done by a single very talented programmer/designer who knows when he's out of his depth[3]. [1] Not "virtualization" per se, but as close to it as makes no difference, if that makes sense? [2] https://www.patreon.com/bcachefs https://www.patreon.com/bcachefs [3] https://bcache.evilpiepirate.org/Encryption/ https://bcache.evilpiepirate.org/Encryption/ (the initial paragraph; AFAIR there's a lot more out there about the process, but I didn't stash any links and that's all I could find in 30 seconds :).

by scribu 9y ago

nwmcsween 9y ago

Not organizational pressures so much but personal politics moreso as there probably quite a few subpar solutions in the kernel because of egos, etc.

taeric 9y ago
Isn't this exactly like in corporations?
- dredmorbius 9y ago
  Interpersonal relations are probably more fluid and less persistant than organisational ones. Organisations are a superset of people, and demonstrate their own emergent properties, most notably organisational persistence beyond the founding remit. People tend not to outlive their founding remit. There's also the (general) case that kernel development doesn't create the self-propogating and self-funding structures you'll find within commercial (or even not-for-profit) organisations. All of which suggests a possibly looser binding strength. The possibility of an alternative development effort tackling a similar problem, under a different group and structure, and being considered not on profitability but on fitness to task likely also helps (modulo inefficiencies in assessing that fitness).

jacquesm 9y ago

> That is to say that subsy[s]tems were defined solely based on technical considerations, which is how it should be if the goal is sound engineering. That never was the goal and if it was it hasn't been achieved. Linux is like democracy, it isn't perfect but it is the best that we've got. Unlike democracy however it is fairly easy to get rid of Linux and replace it with something better.

jacques_chester 9y ago
> Unlike democracy however it is fairly easy to get rid of Linux and replace it with something better. Is it?
- inopinatus 9y ago
  My first thought was "sure, FreeBSD". My second was "hmm, not always so easy".
- jacquesm 9y ago
  Yes. Ironically, the hardest parts would be closed source drivers (Nvidia, several others). Linux after all is only a kernel. One interesting way in which you could do this is to build a wrapper around device drivers so that each device driver becomes a process on top of a microkernel. The userland can mostly remain unchanged as long as you expose the expected interfaces.
  - ifdefdebug 9y ago
    "only a kernel" Come on. You think that is the easy part? Just because everybody can hack up some "kernel" and boot their machine into it? I think you know the relevant Linus Torvalds quote in this context ...
    - jacquesm 9y ago
      I bootstrapped a QnX compatible kernel (with GNU userland) in about 2 years that was then used to power a bunch of special purpose routers. Yes, that's the easy part. As Jacques_Chester already remarked elsewhere in this thread the community part (herding cats) and getting the required buy-in is the hard part. The first can be done, I'm 100% sure of that, the second I'm not so sure. I'm kind of surprised that on HN of all places something like a kernel would be considered somehow beyond re-creation or requiring special god-like powers.
      - ifdefdebug 9y ago

xyzzy_plugh 9y ago

That sounds nice, but realistically a ton of driver code is shoehorned in by sloppy corporate sponsors whose contributors stumble down the narrow, winding line between getting paid and making Linus happy.

lomnakkus 9y ago

> That is to say that subsytems were defined solely based on technical considerations, which is how it should be if the goal is sound engineering. I think that's too idealistic. As another sibling poster pointed out, it's more like a democracy... with all the attendant upsides and downsides. (Also, I'm sure there's a lot of interpersonal politics involved, even if it's all over email.) Don't get me wrong, the Linux kernel works surprisingly well and I rely on it for almost everything (including my livelihood), but if you really look at it, some of the subsystems are shockingly bad. I think a good example is containers/namespaces which have a ridiculously bad security record. (See "user namespaces".) Again, I'm sure the people working on these things had the best of intentions, are very competent generally, and were hampered by the "never break userspace" rule. However, if containers/namespaces were truly designed, it could have been done a lot better. (See "Zones" on Solaris et al.)

X86BSD 9y ago
I agree with you. But I think you're being generous when saying some of the subsystems and architecture are shockingly bad. Some of them are not even designed. If you look at them it's like "This isn't even useable!" The problem with namespaces and all the "container" tooling built on top of it is, as you noted, security. They were never designed to be secure from the start. It was not meant to be. And it was always puzzling to me why someone would use them to build containers on. Knowing it wasn't secure. As you also note Solaris got it right with zones, FreeBSD got it right with jails, there was plenty of options to use and build on. But as is the typical Linux story they had to do things there own way, and poorly. I still have no idea what Linux land is going to do for a filesystem. Since it's pretty clear ZFS is kept at arms length. Btrfs seems like the horse they are hitching to? Filesystems are hard. Which is why they can't seem to write a solid, reliable one and keep starting over. I think it's a big issue for them and is only going to get worse as data continues to explode exponentially. I wish zfs had been BSD licensed. And I wish Linux had simply adopted it as well. But those ships have sailed. Where to now?
- lomnakkus 9y ago
  Agreed, I probably was being a bit too generous, but I usually try to be as "diplomatic" as possible :). I don't know much about the BSDs, but my impression was that jails don't really offer nearly enough to rival Zones. Zones is really "containers done right"[1]. Definitely agree on ZFS. I'm currently using ZoL on my Ubuntu servers, but I'm not going to try using it on anything else. (FWIW, it's working marvellously so far.) Personally, my current best hope for a good Linux FS at the moment is Bcachefs[2]. It's based on the proven Bcache. It's mostly done by a single very talented programmer/designer who knows when he's out of his depth[3]. [1] Not "virtualization" per se, but as close to it as makes no difference, if that makes sense? [2] https://www.patreon.com/bcachefs https://www.patreon.com/bcachefs [3] https://bcache.evilpiepirate.org/Encryption/ https://bcache.evilpiepirate.org/Encryption/ (the initial paragraph; AFAIR there's a lot more out there about the process, but I didn't stash any links and that's all I could find in 30 seconds :).