7 ms·

A huge portion of Linux is drivers and support for different processor architectures. Yes, development was chaotic in the nineties and the code showed. But a lo

by jpfr 9y ago

A huge portion of Linux is drivers and support for different processor architectures. Yes, development was chaotic in the nineties and the code showed. But a lot of engineering effort went into making the core really nice. https://unix.stackexchange.com/a/223763 https://unix.stackexchange.com/a/223763 With regards to POSIX, it is amazing how well this API is holding up. There are quite a few implementions from GNU, BSDs, Microsoft (at least partial support in MSVC) and a few others (e.g. musl). So POSIX support is a given on most systems. Why replace it with something that breaks existing code? https://www.musl-libc.org/faq.html https://www.musl-libc.org/faq.html Not to say there is no bloat. But some bloat is the patina that all succesful systems take on over time. Is the bloat small enough to be managed and/or contained? I say yes.

derefr 9y ago
> So POSIX support is a given on most systems. Why replace it with something that breaks existing code? You're not necessarily breaking existing code. Both macOS and Windows are built on non-POSIX primitives that have POSIX compatibility layers. It seems that the conclusion most of industry has reached is that, whether or not POSIX is a useful API for your average piece of software, there are still better base-layer semantics to architect your kernel, IPC mechanisms, etc. in terms of than the POSIX ones. You can always support a POSIX "flavor" or "branded zone" or "compatibility subsystem" or whatever you want to call it, to run other people's code, after you've written all your code against the nicer set of primitives. An potentially-enlightening analogy: POSIX is like OpenGL. Why do people want Vulkan if OpenGL exists? Well, because Vulkan is a more flexible base-layer with better semantics for high-efficiency use-cases. And if you start with Vulkan, the OpenGL APIs can still be implemented (efficiently!) in terms of them; whereas if you start with an OpenGL-based graphics driver, you can't "get to" (efficient) Vulkan support from there. All that aside, though, I would expect that the real argument is: Fuchsia is for ChromeOS. Google are happy to be the sole maintainers of ChromeOS's kernel and all of its system services, so why not rewrite them all to take advantage of better system-primitive semantics? And Google doesn't have to worry about what apps can run on a Fuchsia-based ChromeOS, because the two ways apps currently run on ChromeOS are "as web-apps in Chrome", or "as Linux ELF executables inside a Linux ABI (or now also Android ABI) sandbox." There is no "ChromeOS software" that needs to be ported to Fuchsia, other than Chrome itself, and the container daemon.
- emn13 9y ago
  Total speculation: but I seriously doubt that Fuchsia is specifically for chromeOS. The whole point of decent, efficient, simple, non-bug-prone APIs is that you probably want to implement pretty much everything on it. Simplicity and low-overhead allow for generality and flexibility. If all you wanted to do was support chromeOS - well, typically you can add hacks even to a messy codebase to support specific usecases. And there are a bunch of linux and ?BSD distros that demonstrate that you can adapt such a system to even very small devices; small enough that there's not much niche left below. Moore's Law/Denard scaling may be comatose on the high-end; but lot's of long-tail stuff is generations behind; which implies that even really low-power IoT stuff that linux is currently ill-suited for will likely be able to run linux without too many tradeoffs. I mean; the original raspberry pi was a 65nm chip@700MHz - that's clearly overkill; and even if chip development never has a breakthrough again, there's clearly a lot of room for those kind of devices to catch up, and a lot of "spare silicon" even in really tiny stuff once you get to small process nodes. But "being able to run linux" doesn't mean it'll be ideal or easy. And efficiency may not be the only issue; security; cost; reliable low latency... there are a whole bunch of things where improvements may be possible. I'm guessing Fuchsia is going to be worse than linux for ChromeOS - in the sense that if ChromeOS really was what google wants it for, they could have gotten better results with linux than they'll be able to get with Fuchsia in the next few years and at a fraction of the cost. Linux just isn't that bad; and a whole new OS including all the interop and user-space and re-education pain is a huge price to pay. But the thing is: if they take that route they may end up with a well tuned linux, but that's it. So my bet is that you'd only ever invest in something like Fuchsia if you're in it for the long run. They're not doing this "for" ChromeOS, even if that may be the first high-profile usage. They're doing this to be enable future savings and quality increases for use cases they probably don't even know they have, yet. In essence: it's a gamble that might pay off in the long run, with some applicability in the medium term - but the medium term alone just doesn't warrant the investment (and risk).
  - __s 9y ago
    Take for example their "replace Intel ME with Linux" project
  - 9y ago
kuschku 9y ago
> So POSIX support is a given on most systems. Why replace it with something that breaks existing code? POSIX says that usernames and uids can overlap, and if a program takes either, it should accept the other a well. And if something could be a username or uid, it should be presumed to be a username, and resolved to a UID. Now assume you have a user "1000" with uid 2000, and a user "2000" with uid 1000... you can see where this goes. And this is why the BSDs have broken POSIX compatibility and require uids to be prefixed with # when they’re used on the CLI.
- JdeBP 9y ago
  No, it does not say that. It says, specifically for certain standard utility programs such as chown and newgrp, that particular program arguments should always be considered to be user names if such user names exist even if they happen to be in the forms of numbers. Nor is what you claim about the BSDs true. Aside from the fact that using the shell's comment character would be markedly inconvenient, especially since the colon is a far better choice for such a marker (c.f. Gerrit Pape's chpst), the OpenBSD and FreeBSD implementations of chown and newgrp do not have such a marker either documented or implemented, and in fact operate as the SUS says. User names and user IDs are, by contrast, explicitly called out by the standard as strings and integers, two quite different things which have no overlap. * http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_436 http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_... * http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_437 http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_...
dmitryminkovsky 9y ago
> But some bloat is the patina that all succesful systems take on over time. What a nice analogy.
- chris_wot 9y ago
  One man’s “bloat” is another man’s feature.
jontro 9y ago
Looks like some parts of POSIX is implemented. See this page for their rationale: https://fuchsia.googlesource.com/docs/+/master/the-book/libc.md https://fuchsia.googlesource.com/docs/+/master/the-book/libc...
naasking 9y ago
> So POSIX support is a given on most systems. Why replace it with something that breaks existing code? Because POSIX has horrible security properties, and does not provide enough guarantees to create truly robust software. See for instance, the recent article on how you simply cannot implement atomic file operations in POSIX -- sqlite has to jump through 1,000 hoops to get something pretty robust, but it shouldn't be this way.
- rurban 9y ago
  The biggest problem is the blocking IO. With async throughout you can design a proper concurrent system.
  - dingo_bat 9y ago
    Is there an example of an os which doesn't have blocking io? To me it seems that blocking io will be needed at some level. You can of course put wrappers around it and present it as async. But in many applications you want to go as low as possible, and in my imagination blocking calls will be the lowest.
    - zzzcpan 9y ago
      It's the other way around. Low level is all async everywhere, blocking is sort of just telling the kernel to wait for async op to complete before running the process.
    - rurban 9y ago
      Midora, Singularity in an OS. Pony in a language. The goal is to avoid blocking, not to offer async also. That's a lost cause. Fuchsia once had the goal to offer async only, but then some manager decided that he needs some blocking, and then it was gone. non-blocking only is lower level than blocking. You can always wait indefinitely for that callback, but usually you have a default timeout. L4 for example offers that API.
  - zaarn 9y ago
    The Linux kernel has Async IO. For a while now. You don't need async for a proper concurrent system. Systems have been concurrent before async IO. The trick is that when a process does an IO you yield it and put it on a wait list and run literally anything else until the kernel receives the OK from the hardware control and resumes the process. Using Async IO in Linux merely means your specific thread won't be immediately suspended until you get data (or it's in a reasonably close cache) It would be quite silly if Linux would wait for every IO synchronously, any single core system would immediately grind to a halt.
StillBored 9y ago
> So POSIX support is a given on most systems. Why replace it with something that breaks existing code? POSIX is a lowest common denominator API, its under specified and loosely interpreted. Which follows from its initial goals, which was basically to specify the bits common between various UNIX implementations. Those implementations obviously didn't want to change their implementation to match a rigid standard, so a lot of wiggle room exists in the standard. The end result is that is pretty much useless for anything beyond "hello world" kinds of applications, both in terms of portability, as well as actual behavior (I could list a lot of cases, but lets leave that to google, with the starting idea to look at a couple posix API's, say close()'s errno's and the differing cases and what causes them on different OS's/filesystems). That is why there isn't a single OS out there that is _ONLY_ POSIX compliant. You need look no further than the 15 year old https://personal.opengroup.org/~ajosey/tr28-07-2003.txt https://personal.opengroup.org/~ajosey/tr28-07-2003.txt and consider that the gap has widened as more performance or security oriented core API's have been introduced and linux's POSIX layer is further refined upon them. Plus, the core API's in no way reflect the hard reality of modern hardware, leaving the standard even more under specified in the case of threads and async IO, which have been poorly bolted on. Then there are all the bits everyone ignores, like the bits about the posix shell (ksh), and how certain utilities behave, while completely ignoring important things like determining metadata about the hardware one is running on. I leave you with: https://stackoverflow.com/questions/2693948/how-do-i-retrieve-the-number-of-processors-on-c-linux https://stackoverflow.com/questions/2693948/how-do-i-retriev... and https://stackoverflow.com/questions/150355/programmatically-find-the-number-of-cores-on-a-machine https://stackoverflow.com/questions/150355/programmatically-... Which is pretty basic information about a modern machine.
- pjmlp 9y ago
  > POSIX is a lowest common denominator API,... Only useful for CLI tty old style apps and daemons.
temac 9y ago
> But a lot of engineering effort went into making the core really nice. I quite like Linux but I would not say the current state of the core kernel is "really" nice. Somewhat nice, maybe, but "really" nice? You just have to look around to see exemples of non-nice things (too much things in some headers, jumping between source files back and forth for no reason, excessive use of hand coded vtables, internal "framework" that are not as simple as they should, at other time lack of abstractions and excessive access to internals of structures, etc). Granted, it is way less buggy than some other softwares, but I think I'll know when I read a really nice core, and for now I've never had the feeling that Linux has one. Still find it quite good, to be clear.
- gnulinux 9y ago
  FreeBSD?
dralley 9y ago
>Why replace it with something that breaks existing code? https://github.com/mpv-player/mpv/commit/1e70e82baa9193f6f027338b0fab0f5078971fbe https://github.com/mpv-player/mpv/commit/1e70e82baa9193f6f02...

by jpfr 9y ago

derefr 9y ago

> So POSIX support is a given on most systems. Why replace it with something that breaks existing code? You're not necessarily breaking existing code. Both macOS and Windows are built on non-POSIX primitives that have POSIX compatibility layers. It seems that the conclusion most of industry has reached is that, whether or not POSIX is a useful API for your average piece of software, there are still better base-layer semantics to architect your kernel, IPC mechanisms, etc. in terms of than the POSIX ones. You can always support a POSIX "flavor" or "branded zone" or "compatibility subsystem" or whatever you want to call it, to run other people's code, after you've written all your code against the nicer set of primitives. An potentially-enlightening analogy: POSIX is like OpenGL. Why do people want Vulkan if OpenGL exists? Well, because Vulkan is a more flexible base-layer with better semantics for high-efficiency use-cases. And if you start with Vulkan, the OpenGL APIs can still be implemented (efficiently!) in terms of them; whereas if you start with an OpenGL-based graphics driver, you can't "get to" (efficient) Vulkan support from there. All that aside, though, I would expect that the real argument is: Fuchsia is for ChromeOS. Google are happy to be the sole maintainers of ChromeOS's kernel and all of its system services, so why not rewrite them all to take advantage of better system-primitive semantics? And Google doesn't have to worry about what apps can run on a Fuchsia-based ChromeOS, because the two ways apps currently run on ChromeOS are "as web-apps in Chrome", or "as Linux ELF executables inside a Linux ABI (or now also Android ABI) sandbox." There is no "ChromeOS software" that needs to be ported to Fuchsia, other than Chrome itself, and the container daemon.

emn13 9y ago
Total speculation: but I seriously doubt that Fuchsia is specifically for chromeOS. The whole point of decent, efficient, simple, non-bug-prone APIs is that you probably want to implement pretty much everything on it. Simplicity and low-overhead allow for generality and flexibility. If all you wanted to do was support chromeOS - well, typically you can add hacks even to a messy codebase to support specific usecases. And there are a bunch of linux and ?BSD distros that demonstrate that you can adapt such a system to even very small devices; small enough that there's not much niche left below. Moore's Law/Denard scaling may be comatose on the high-end; but lot's of long-tail stuff is generations behind; which implies that even really low-power IoT stuff that linux is currently ill-suited for will likely be able to run linux without too many tradeoffs. I mean; the original raspberry pi was a 65nm chip@700MHz - that's clearly overkill; and even if chip development never has a breakthrough again, there's clearly a lot of room for those kind of devices to catch up, and a lot of "spare silicon" even in really tiny stuff once you get to small process nodes. But "being able to run linux" doesn't mean it'll be ideal or easy. And efficiency may not be the only issue; security; cost; reliable low latency... there are a whole bunch of things where improvements may be possible. I'm guessing Fuchsia is going to be worse than linux for ChromeOS - in the sense that if ChromeOS really was what google wants it for, they could have gotten better results with linux than they'll be able to get with Fuchsia in the next few years and at a fraction of the cost. Linux just isn't that bad; and a whole new OS including all the interop and user-space and re-education pain is a huge price to pay. But the thing is: if they take that route they may end up with a well tuned linux, but that's it. So my bet is that you'd only ever invest in something like Fuchsia if you're in it for the long run. They're not doing this "for" ChromeOS, even if that may be the first high-profile usage. They're doing this to be enable future savings and quality increases for use cases they probably don't even know they have, yet. In essence: it's a gamble that might pay off in the long run, with some applicability in the medium term - but the medium term alone just doesn't warrant the investment (and risk).
- __s 9y ago
  Take for example their "replace Intel ME with Linux" project
- 9y ago

kuschku 9y ago

> So POSIX support is a given on most systems. Why replace it with something that breaks existing code? POSIX says that usernames and uids can overlap, and if a program takes either, it should accept the other a well. And if something could be a username or uid, it should be presumed to be a username, and resolved to a UID. Now assume you have a user "1000" with uid 2000, and a user "2000" with uid 1000... you can see where this goes. And this is why the BSDs have broken POSIX compatibility and require uids to be prefixed with # when they’re used on the CLI.

JdeBP 9y ago
No, it does not say that. It says, specifically for certain standard utility programs such as chown and newgrp, that particular program arguments should always be considered to be user names if such user names exist even if they happen to be in the forms of numbers. Nor is what you claim about the BSDs true. Aside from the fact that using the shell's comment character would be markedly inconvenient, especially since the colon is a far better choice for such a marker (c.f. Gerrit Pape's chpst), the OpenBSD and FreeBSD implementations of chown and newgrp do not have such a marker either documented or implemented, and in fact operate as the SUS says. User names and user IDs are, by contrast, explicitly called out by the standard as strings and integers, two quite different things which have no overlap. * http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_436 http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_... * http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_437 http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_...

dmitryminkovsky 9y ago

> But some bloat is the patina that all succesful systems take on over time. What a nice analogy.

chris_wot 9y ago
One man’s “bloat” is another man’s feature.

jontro 9y ago

Looks like some parts of POSIX is implemented. See this page for their rationale: https://fuchsia.googlesource.com/docs/+/master/the-book/libc.md https://fuchsia.googlesource.com/docs/+/master/the-book/libc...

naasking 9y ago

> So POSIX support is a given on most systems. Why replace it with something that breaks existing code? Because POSIX has horrible security properties, and does not provide enough guarantees to create truly robust software. See for instance, the recent article on how you simply cannot implement atomic file operations in POSIX -- sqlite has to jump through 1,000 hoops to get something pretty robust, but it shouldn't be this way.

rurban 9y ago
The biggest problem is the blocking IO. With async throughout you can design a proper concurrent system.
- dingo_bat 9y ago
  Is there an example of an os which doesn't have blocking io? To me it seems that blocking io will be needed at some level. You can of course put wrappers around it and present it as async. But in many applications you want to go as low as possible, and in my imagination blocking calls will be the lowest.
  - zzzcpan 9y ago
    It's the other way around. Low level is all async everywhere, blocking is sort of just telling the kernel to wait for async op to complete before running the process.
  - rurban 9y ago
    Midora, Singularity in an OS. Pony in a language. The goal is to avoid blocking, not to offer async also. That's a lost cause. Fuchsia once had the goal to offer async only, but then some manager decided that he needs some blocking, and then it was gone. non-blocking only is lower level than blocking. You can always wait indefinitely for that callback, but usually you have a default timeout. L4 for example offers that API.
- zaarn 9y ago
  The Linux kernel has Async IO. For a while now. You don't need async for a proper concurrent system. Systems have been concurrent before async IO. The trick is that when a process does an IO you yield it and put it on a wait list and run literally anything else until the kernel receives the OK from the hardware control and resumes the process. Using Async IO in Linux merely means your specific thread won't be immediately suspended until you get data (or it's in a reasonably close cache) It would be quite silly if Linux would wait for every IO synchronously, any single core system would immediately grind to a halt.

StillBored 9y ago

> So POSIX support is a given on most systems. Why replace it with something that breaks existing code? POSIX is a lowest common denominator API, its under specified and loosely interpreted. Which follows from its initial goals, which was basically to specify the bits common between various UNIX implementations. Those implementations obviously didn't want to change their implementation to match a rigid standard, so a lot of wiggle room exists in the standard. The end result is that is pretty much useless for anything beyond "hello world" kinds of applications, both in terms of portability, as well as actual behavior (I could list a lot of cases, but lets leave that to google, with the starting idea to look at a couple posix API's, say close()'s errno's and the differing cases and what causes them on different OS's/filesystems). That is why there isn't a single OS out there that is _ONLY_ POSIX compliant. You need look no further than the 15 year old https://personal.opengroup.org/~ajosey/tr28-07-2003.txt https://personal.opengroup.org/~ajosey/tr28-07-2003.txt and consider that the gap has widened as more performance or security oriented core API's have been introduced and linux's POSIX layer is further refined upon them. Plus, the core API's in no way reflect the hard reality of modern hardware, leaving the standard even more under specified in the case of threads and async IO, which have been poorly bolted on. Then there are all the bits everyone ignores, like the bits about the posix shell (ksh), and how certain utilities behave, while completely ignoring important things like determining metadata about the hardware one is running on. I leave you with: https://stackoverflow.com/questions/2693948/how-do-i-retrieve-the-number-of-processors-on-c-linux https://stackoverflow.com/questions/2693948/how-do-i-retriev... and https://stackoverflow.com/questions/150355/programmatically-find-the-number-of-cores-on-a-machine https://stackoverflow.com/questions/150355/programmatically-... Which is pretty basic information about a modern machine.

pjmlp 9y ago
> POSIX is a lowest common denominator API,... Only useful for CLI tty old style apps and daemons.

temac 9y ago

> But a lot of engineering effort went into making the core really nice. I quite like Linux but I would not say the current state of the core kernel is "really" nice. Somewhat nice, maybe, but "really" nice? You just have to look around to see exemples of non-nice things (too much things in some headers, jumping between source files back and forth for no reason, excessive use of hand coded vtables, internal "framework" that are not as simple as they should, at other time lack of abstractions and excessive access to internals of structures, etc). Granted, it is way less buggy than some other softwares, but I think I'll know when I read a really nice core, and for now I've never had the feeling that Linux has one. Still find it quite good, to be clear.

gnulinux 9y ago
FreeBSD?

dralley 9y ago

>Why replace it with something that breaks existing code? https://github.com/mpv-player/mpv/commit/1e70e82baa9193f6f027338b0fab0f5078971fbe https://github.com/mpv-player/mpv/commit/1e70e82baa9193f6f02...