36 ms·

> That is to say that subsytems were defined solely based on technical considerations, which is how it should be if the goal is sound engineering. I think that

> That is to say that subsytems were defined solely based on technical considerations, which is how it should be if the goal is sound engineering. I think that's too idealistic. As another sibling poster pointed out, it's more like a democracy... with all the attendant upsides and downsides. (Also, I'm sure there's a lot of interpersonal politics involved, even if it's all over email.) Don't get me wrong, the Linux kernel works surprisingly well and I rely on it for almost everything (including my livelihood), but if you really look at it, some of the subsystems are shockingly bad. I think a good example is containers/namespaces which have a ridiculously bad security record. (See "user namespaces".) Again, I'm sure the people working on these things had the best of intentions, are very competent generally, and were hampered by the "never break userspace" rule. However, if containers/namespaces were truly designed, it could have been done a lot better. (See "Zones" on Solaris et al.)

X86BSD 9y ago
I agree with you. But I think you're being generous when saying some of the subsystems and architecture are shockingly bad. Some of them are not even designed. If you look at them it's like "This isn't even useable!" The problem with namespaces and all the "container" tooling built on top of it is, as you noted, security. They were never designed to be secure from the start. It was not meant to be. And it was always puzzling to me why someone would use them to build containers on. Knowing it wasn't secure. As you also note Solaris got it right with zones, FreeBSD got it right with jails, there was plenty of options to use and build on. But as is the typical Linux story they had to do things there own way, and poorly. I still have no idea what Linux land is going to do for a filesystem. Since it's pretty clear ZFS is kept at arms length. Btrfs seems like the horse they are hitching to? Filesystems are hard. Which is why they can't seem to write a solid, reliable one and keep starting over. I think it's a big issue for them and is only going to get worse as data continues to explode exponentially. I wish zfs had been BSD licensed. And I wish Linux had simply adopted it as well. But those ships have sailed. Where to now?
- lomnakkus 9y ago
  Agreed, I probably was being a bit too generous, but I usually try to be as "diplomatic" as possible :). I don't know much about the BSDs, but my impression was that jails don't really offer nearly enough to rival Zones. Zones is really "containers done right"[1]. Definitely agree on ZFS. I'm currently using ZoL on my Ubuntu servers, but I'm not going to try using it on anything else. (FWIW, it's working marvellously so far.) Personally, my current best hope for a good Linux FS at the moment is Bcachefs[2]. It's based on the proven Bcache. It's mostly done by a single very talented programmer/designer who knows when he's out of his depth[3]. [1] Not "virtualization" per se, but as close to it as makes no difference, if that makes sense? [2] https://www.patreon.com/bcachefs https://www.patreon.com/bcachefs [3] https://bcache.evilpiepirate.org/Encryption/ https://bcache.evilpiepirate.org/Encryption/ (the initial paragraph; AFAIR there's a lot more out there about the process, but I didn't stash any links and that's all I could find in 30 seconds :).
  - X86BSD 9y ago
    Zones was the logical conclusion to FreeBSD jail work. They added the full features jails now has finished. Each zone getting its own IP stack for instance. Jails didn't have that in the beginning. It was written by one guy to simply offer a basic but secure container for running multiple web servers in isolation from the host. Had he had the resources of sun Microsystems I'm sure he would have had all the features zones had from the start. But one man can only do so much with finite resources. And the main design goal was security and isolation. Which was accomplished. Separate network stacks was added later. I'm glad to hear your using it. I really can't imagine using anything else. Open source or commercial. Nothing is as compelling or reliable and easy to use. Me personally, i am forever grateful for Sun open sourcing it. I remember pre zfs days, with volume managers, and how hard it was to grow a ufs/ffs anything, just an absolute nightmare. And backups? Ugh. Tapes. Hoping every bit was copied, but never really knowing for sure. Just ugh. The pain. I've heard bcachefs talked about to. And as you originally stated I think these folks writing btrfs and bcachefs are talented and are doing the best they can but again FS's are hard. Look at all the garbage we've been stuck with from all the major commercial vendors until ZFS. Everything Microsoft puked up, or Apple, even the major UNIX vendors sucked at making file systems and volume management. Veritas? I'd rather stab my eyes with soldering irons than use any of that stuff ever again. Seriously ZFS compared to all of them is night and day. I'll never be able to thank the developers enough for helping me out of the pit of filesystem and volume management hell. I'll be interested to see how bcachefs ends up. But mostly I think everyone will be trying to play catch up to ZFS and more than likely doing it poorly. But I applaud their efforts to try! Thanks for the links btw, I always enjoy reading up on various tech.
lawpoop 9y ago
I wonder if there's any form of human organization capable of producing-- and more importantly, /maintaining/-- well designed, well-thought out complex system. I suspect that even a NASA or an Apple or a Google or some other social structure would make a product with some painfully bad parts lurking somewhere in it. I further suspect that the difference is, with open source code, the "general" public is actually able to become aware of it, rather than just the engineers working on it.
- lomnakkus 9y ago
  I think we have done some complex systems well. Just not very many computer systems. It may be a consequence of computer systems being too pliable. EDIT: Actually, before I forget, I heard a great aphorism the other day: When designing (distributed) computer systems, try designing it as if each "node" were an actual human. That'll usually ferret out most of the flaws in your reasoning. (No, it's obviously not foolproof, it's basically just a sanity check against assuming absolute reliability, etc. Kill cats. It's a rhetorical device meant to make you think. So think. Hey, do you own thinking. I can't think for all of you. Live your own lives!)
  - Pica_soO 9y ago
    A well done complex system, turns invisible, because complexity is severely reduced wherever you can. I know we engineers are getting high on it, but if you remove complex components, everyone else is much happier. Really good complex systems thus, are seemingly simple systems, that cover with stupid components a suprisingly big number of uses-cases.
  - lawpoop 9y ago
    Oh, I agree, there's probably several /systems/ that are all-around good. My argument is that there is no organizational structure that is more likely to product and maintain such a system. People identify the bad parts of an open-source project and say "See? This is why we need Serious Engineers|Business Acumen|etc". Meanwhile, the bad parts of private project simply aren't made public.

X86BSD 9y ago

I agree with you. But I think you're being generous when saying some of the subsystems and architecture are shockingly bad. Some of them are not even designed. If you look at them it's like "This isn't even useable!" The problem with namespaces and all the "container" tooling built on top of it is, as you noted, security. They were never designed to be secure from the start. It was not meant to be. And it was always puzzling to me why someone would use them to build containers on. Knowing it wasn't secure. As you also note Solaris got it right with zones, FreeBSD got it right with jails, there was plenty of options to use and build on. But as is the typical Linux story they had to do things there own way, and poorly. I still have no idea what Linux land is going to do for a filesystem. Since it's pretty clear ZFS is kept at arms length. Btrfs seems like the horse they are hitching to? Filesystems are hard. Which is why they can't seem to write a solid, reliable one and keep starting over. I think it's a big issue for them and is only going to get worse as data continues to explode exponentially. I wish zfs had been BSD licensed. And I wish Linux had simply adopted it as well. But those ships have sailed. Where to now?

lomnakkus 9y ago
Agreed, I probably was being a bit too generous, but I usually try to be as "diplomatic" as possible :). I don't know much about the BSDs, but my impression was that jails don't really offer nearly enough to rival Zones. Zones is really "containers done right"[1]. Definitely agree on ZFS. I'm currently using ZoL on my Ubuntu servers, but I'm not going to try using it on anything else. (FWIW, it's working marvellously so far.) Personally, my current best hope for a good Linux FS at the moment is Bcachefs[2]. It's based on the proven Bcache. It's mostly done by a single very talented programmer/designer who knows when he's out of his depth[3]. [1] Not "virtualization" per se, but as close to it as makes no difference, if that makes sense? [2] https://www.patreon.com/bcachefs https://www.patreon.com/bcachefs [3] https://bcache.evilpiepirate.org/Encryption/ https://bcache.evilpiepirate.org/Encryption/ (the initial paragraph; AFAIR there's a lot more out there about the process, but I didn't stash any links and that's all I could find in 30 seconds :).
- X86BSD 9y ago
  Zones was the logical conclusion to FreeBSD jail work. They added the full features jails now has finished. Each zone getting its own IP stack for instance. Jails didn't have that in the beginning. It was written by one guy to simply offer a basic but secure container for running multiple web servers in isolation from the host. Had he had the resources of sun Microsystems I'm sure he would have had all the features zones had from the start. But one man can only do so much with finite resources. And the main design goal was security and isolation. Which was accomplished. Separate network stacks was added later. I'm glad to hear your using it. I really can't imagine using anything else. Open source or commercial. Nothing is as compelling or reliable and easy to use. Me personally, i am forever grateful for Sun open sourcing it. I remember pre zfs days, with volume managers, and how hard it was to grow a ufs/ffs anything, just an absolute nightmare. And backups? Ugh. Tapes. Hoping every bit was copied, but never really knowing for sure. Just ugh. The pain. I've heard bcachefs talked about to. And as you originally stated I think these folks writing btrfs and bcachefs are talented and are doing the best they can but again FS's are hard. Look at all the garbage we've been stuck with from all the major commercial vendors until ZFS. Everything Microsoft puked up, or Apple, even the major UNIX vendors sucked at making file systems and volume management. Veritas? I'd rather stab my eyes with soldering irons than use any of that stuff ever again. Seriously ZFS compared to all of them is night and day. I'll never be able to thank the developers enough for helping me out of the pit of filesystem and volume management hell. I'll be interested to see how bcachefs ends up. But mostly I think everyone will be trying to play catch up to ZFS and more than likely doing it poorly. But I applaud their efforts to try! Thanks for the links btw, I always enjoy reading up on various tech.

lawpoop 9y ago

I wonder if there's any form of human organization capable of producing-- and more importantly, /maintaining/-- well designed, well-thought out complex system. I suspect that even a NASA or an Apple or a Google or some other social structure would make a product with some painfully bad parts lurking somewhere in it. I further suspect that the difference is, with open source code, the "general" public is actually able to become aware of it, rather than just the engineers working on it.

lomnakkus 9y ago
I think we have done some complex systems well. Just not very many computer systems. It may be a consequence of computer systems being too pliable. EDIT: Actually, before I forget, I heard a great aphorism the other day: When designing (distributed) computer systems, try designing it as if each "node" were an actual human. That'll usually ferret out most of the flaws in your reasoning. (No, it's obviously not foolproof, it's basically just a sanity check against assuming absolute reliability, etc. Kill cats. It's a rhetorical device meant to make you think. So think. Hey, do you own thinking. I can't think for all of you. Live your own lives!)
- Pica_soO 9y ago
  A well done complex system, turns invisible, because complexity is severely reduced wherever you can. I know we engineers are getting high on it, but if you remove complex components, everyone else is much happier. Really good complex systems thus, are seemingly simple systems, that cover with stupid components a suprisingly big number of uses-cases.
- lawpoop 9y ago
  Oh, I agree, there's probably several /systems/ that are all-around good. My argument is that there is no organizational structure that is more likely to product and maintain such a system. People identify the bad parts of an open-source project and say "See? This is why we need Serious Engineers|Business Acumen|etc". Meanwhile, the bad parts of private project simply aren't made public.