6 ms·

Ideal OS: Rebooting the Desktop Operating System (2017)

83 points by mmphosis 2y ago

HKH2 2y ago
Why not just build your OS on top of Linux? Then you can get driver support at least.
- goodthenandnow 2y ago
  Because Linux, the kernel, as well as other Unixes, don't have the right abstractions to easily build these. And there's all the old cruft. Just look at what Android had to do to implement a secure platform with somewhat of a well defined non-1980's OS API - it basically removed the Unix part of the OS and built a Microkernel-like thing on top of it. Driver support would be a very important part though.
  - kimixa 2y ago
    > Just look at what Android had to do to implement a secure platform with somewhat of a well defined non-1980's OS API - it basically removed the Unix part of the OS and built a Microkernel-like thing on top of it. ...Huh? Pretty much all Android adds to the kernel is Binder, which is really a performance optimization for it's version of IPC, which today has equivalents that mean it probably wouldn't need it's own version now if it didn't need compatibility. The security model allows native apps, which have access to all the normal libc, or even direct syscalls. Some hardware access is marshalled through userspace daemons, which may implement security for things like camera/microphone access, but that's been a pretty common thing on other unix-style OSs for ages. Though the separation of each app into it's own UID might not have been the original intent, but it seems to work implemented with those same unix capabilities. Yeah, most apps probably don't use POSIX APIs directly, and instead the java Android APIs, but they are very much build on top of the "crufty 1980s OS APIs", rather than bypassing them.
    - goodthenandnow 2y ago
      Android blocks most of Linux syscalls with seccomp. They make heavy usage of SELinux. Binder is not just a simple IPC system. It requires userspace parts which aren't even the parties who are talking through IPC. Like you said it has lots of performance optimizations, and that requires it hooking up to a number of kernel subsystems. At the beginning it was met with a lot of resistance from kernel maintarners when tried to be merged. Why did Google came up with it? When talking about the 'Android sandbox' people usually talk about how Android gives each app an different uid/gid. But that's just the beginning of the story. It wasn't enough for sandboxing needs since the beginning. Thats why they took the binder idea from BeOS and used on Android. Since then Google has blocked everything. Native code is needed on any practical system, but today's native code has access mostly to nothing, just the common needs to run (like libc which is used due backward compatibility). The proper way to get something is through Binder (and even that is fine-grained controlled on a per process basis). Linux and Unixes in general don't implement true capabilities. Android does through Binder, but since Binder lives “inside Linux” everything else has to be blocked to make these capabilities useful. > Some hardware access is marshalled through userspace daemons Most of stuff are, at least initially, orchestrated through userspace software - daemons like you said, but mostly HALs (which are also daemons). The main takeaway is that binder is _the core_ of Android userspace. It's tightly integrated with SELinux and everything passes through it at some point. Permissions are enforced through it. In the end, the whole “binder subsystem” is architectured like a Microkernel system.
      - kimixa 2y ago
        I'd argue that permissions aren't really enforced by binder, just binder is used to talk to the endpoints that enforce permissions. Binder has no interaction with SELinux outside the policy requirements to allow apps to open it's device nodes - there's no interaction in the kernel code at all [0]. It's just an opaque data blob IPC mechanism, not tagging payloads with capabilities or similar in-kernel. At the point that binder was created, there wasn't a good kernel-level IPC system integrated into linux, true, but I'm not sure I agree with the idea that it's in any way tied into the capabilities/permissions system outside from being a communication channel to the userspace daemons things that do. And native code does have access to every service on the device - just as any app. It's just more painful than the Java layer, and some things aren't guaranteed to be stable over API versions. Less useful for apps, but still not part of the security model. At an extreme level, the java code is run in the same memory address space as any native code, there's no protection from that. Hell, some apps even used to patch dalvik/ART live. It was insane and fragile, but entirely possible. And the "disallowed" syscalls are those not exposed by libc though some more are restricted for user apps (mostly around UID management, which is the big "major" deviation from standard unix, and system stuff like loading kernel modules or rebooting...) [1] So maybe it doesn't allow some linux extensions, or could remove the "old" versions of replaced syscalls (when arguments were found not to be large enough, flags arguments added etc.). But there's not really anything missing from there as used in 99% of unix userspace programs anyway. So I maintain it's still a system build on top of a unix kernel and libc with a few additions on the side, and all that is still accessible to apps. Perhaps you think I'm including all the stuff people associate with modern "Desktop Linux"? Like X11 and/or wayland? Or the entire FHS? That's the main thing Android has completely divorced from, but that also isn't "Unix", or even really from the 80s even if it has some ancestors there. [0] https://android.googlesource.com/kernel/common/+/refs/heads/android-mainline/drivers/android/ https://android.googlesource.com/kernel/common/+/refs/heads/... [1] See SECCOMP*.txt and SYSCALLS.txt from https://android.googlesource.com/platform/bionic/+/refs/heads/main/libc/ https://android.googlesource.com/platform/bionic/+/refs/head...
  - theamk 2y ago
    Eh the system described in the post is mostly about taking away (no filesystem access, no multiple IPC methods, no X11...) All new required software seems to be only userspace: - New IPC system: presumably it's abstracted, so can use any existing IPC for implementation. Or, purely as optimization, use Android's Binder or bring back bus1 (it was such a great idea) - Document Database: pure userspace; in the most extreme case use raw block device - Compositor / WM / Windows: this was always userspace already - Rebuild the apps: userspace as well Then once you new fancy APIs are ready, you use seccomp and/or per-app users and/or namespaces to isolate apps from everything except the few things people want.
    - goodthenandnow 2y ago
      > the system described in the post is mostly about taking away I agree. The hard part is, as usual, on the details. How do you take away stuff so that you still end up with an usable general purpose OS? That's definitely not easy, specially if you want to make it in a proper way, without hacking things up. For example, suppose you make the decision to use the Linux kernel, which is a good one - you get driver support, networking, filesystems, and all the good stuff and general primitives for a common OS of today.. Given that what one wants is to remove stuff, in order to make that secure (which is requirement) you'd need to isolate and block everything because on Linux everything is accessible to a process by default, and the common primitives for control are very basic - you'd have to do a _lot_ of work in userspace to block it. It would contaminate your userspace architecture with details and stuff. And then you'd probably end up needing something to make the userspace parts able to communicate with each other, like Binder on Androids case (see my other comment in this thread).
aredox 2y ago
The state of desktop paradigms has been one of great stagnation, if not outright regression. The VR fad is never going anywhere: its problems are an order of magnitude worse and engineers and companies nowadays seem unable to do basic 2D interface right.
- m463 2y ago
  I really notice this when I play PC games. They are easy to get into, they are highly customizable, and they teach you bit by bit until you're pretty powerful at doing things.
  - paulryanrogers 2y ago
    Each game though is super narrow, and is built on OS primitives already established. An OS must be all things to all users and all devs, so some complexity and bloat are inevitable. Users don't want to start from scratch to learn every program they need. Also, they often bounce off of games with controls outside their comfort zone.
- linguae 2y ago
  I agree. In my opinion, the problem is that Microsoft and Apple discovered more lucrative markets, making their desktops less important. For Apple, the iPhone’s revenue completely dwarfs the Mac’s, and even services bring in more revenue than the Mac. Microsoft still makes a lot of money selling Windows licenses to OEMs and corporations, but somehow Microsoft still feels compelled to extract even more money from Windows by placing ads in the OS and by selling subscription services. As a fan of desktop GUI software that conforms to well-developed platform UI guidelines such as the old Apple Human Interface Guidelines, I believe the rise of mobile computing and the Web hurt the desktop. Software companies ship cross-platform applications using least-common-denominator frameworks instead of writing apps with UIs that are tailor-made to each platform. While this saves development time, and while it’s popular to say “having an Electron app is better than having no app,” it makes the desktop experience less consistent and more difficult to use. In addition, the desktop UI has become more mobile-like even in ways that are inappropriate for desktop computing, such as the proliferation of hamburger windows, the tendency to hide settings (thus requiring additional clicks to perform operations that used to be done in one or two clicks), oversized bars, title bars being crammed with buttons (making it harder to move windows via dragging the title bar), excessive padding, etc. There is a difference between making things easier for users versus dumbing things down, but I feel that a lot of software designers these days don’t know the difference. I would like to see a return to UI consistency across applications, and I’d also like to see a desktop environment that caters to power users. A dream of mine is some type of “component-based” desktop operating system, where everything is an object that can be manipulated through either a GUI, the command line, or programmatically. The backend would be inspired by technologies like Smalltalk, GNU Emacs, Lisp machines, and Apple OpenDoc, but with modern considerations such as security and parallelism.
Qxliang 2y ago
Is getting rid of command line interfaces really a good idea?I don't think so,a lot of people work with and are fond of the CLI tool.
- theendisney 2y ago
  Depends entirely on what you replace it with. A fun thought, key bindings and task visualisations could mimic cli
- al_borland 2y ago
  If the CLI were to go away, the OS would still need to have rich APIs for automation. While it may not be something the average user ever uses or knows about, it’s vital for power users and the enterprise.
  - jcelerier 2y ago
    And then you want these automation APIs to be accessible through a REPL for quick prototyping and...
  - vmfunction 2y ago
    Either that or the OS need to provide an API to allow major languages to be able to interact with the system other than the GUI. sh can be an add on.
    - goodthenandnow 2y ago
      Not sure if this is what you meant, but I'd like to expand a little bit on it. - This API should be programming language agnostic (like, with a well defined ABI). Ideally an IPC based one, with a well defined wire format and protocols. - Not a singular API, but an “API system” with extensibility taken into account from the beginning (with things like versioning). It should be able to support arbitrary new API definitions so that other apps could consume, and also produce. - Such an API system should be used by the OS itself too, of course. So the OS would have the resulting benefits and the whole system would be more uniform.
janice1999 2y ago
This article predates Apple's visionOS. I wonder what the author's take on it would be.
- joshmarinacci 2y ago
  I love Apple’s hardware and the metaphors are well thought out, but I’m disappointed that one of the best design companies in the world couldn’t come up with better uses than photos and having a virtual Mac monitor.
  - pjmlp 2y ago
    Because they didn't want it to be another MetaQuest or HoloLens, thus there isn't much left as use cases.
GalaxyNova 2y ago
[flagged]
linguae 2y ago
This is a nice document, and many of the suggestions dovetail with my own ideas for a component-based operating system, which I wrote about in 2020 (http://mmcthrow-musings.blogspot.com/2020/04/a-proposal-for-flexible-composable.html http://mmcthrow-musings.blogspot.com/2020/04/a-proposal-for-...) but haven’t had the time to work on. I wonder if there’s been any follow-up work on Ideal OS?
- zzo38computer 2y ago
  Some of my ideas, also. I agree with the four items listed in "Problems with Today's Desktop Environments and Applications". However, these are not the only issues. (I even thing the older Windows 3.1 and even older UI were well enough; you don't need such fancy graphics.) Your "Monolithic Applications versus Composable Tools" also is reasonable. What you list about Unix philsophy are the good ideas of Unix philsophy; it is too bad that many modern programs don't support it. I would allow command-line and GUI and easily interact between them. My idea involves capability-based security with proxy capabilities, and a program can send/receive/wait with capabilities it has access to; a message send/received consists of bytes and/or capabilities. (It is also similar than actor model in some ways.) This is what the kernel does (and programs can create their own capabilities called "proxy capabilities"); all I/O is required to use capabilities. (When a program starts, it must receive an initial message containing some capabilities, so that it can do I/O.) My idea also is to make the format which will be called "Common Data Format"; it can store documents, databases, and other stuff, and can be used in command-line and GUI. (It also involves some of the ideas of OpenDoc, and of TRON Application Databus, but is different from both of them.) There are also many other considerations: hypertext file system, window indicators, security, TRON character code, network transparency, emulation, etc. About "Not a New Operating System", my idea would be that the specification can be done and implementations of the individual parts can be done independently and interchangeable, and that an implementation can be made to run stand-alone or inside of another operating system (something that Inferno also does); this can avoid the problems of drivers at first, while when it is made stand-alone, drivers can be made for it in that way, too. However, I do not intend to include a web browser by default. (For making such a thing on Linux, I don't know exactly what capsicum blocks; capabilities are supposed to required for all I/O including accessing the current date/time. My system idea is similar like if you have only socketpair to create new channels of communication, and being able to send/receive messages that may contain SCM_RIGHTS, and the ability to wait for objects.)
deathanatos 2y ago
Part of me wonders how much of this is hardware vendors releasing unusable hardware. I don't know what the hangup on the RPi is (that seems odd, given how prevalent Linux on the RPi is) but that & the keyboard RGB LEDs seems like your bog standard "the hardware can't or if it can, the vendor didn't bother to document anything". The FOSS world has done an incredible amount of work reverse engineering over the years, but the behavior of HW companies are more to blame, and that's not a problem you'll solve with tech. But hardware vendors have for years, and still do, just say "our use case, or the highway, you don't really own the thing you're buying." And there's not much out there for one to "vote with one's wallet" on it. "FS can't store a file in two spots" … I want to accuse OP of being a Windows user, but they're obviously not [only]. So that's … an odd critique? I still think hierarchical+symlinks beats trying to tag stuff, and I've lived through enough decades of AI trying to tag stuff and deciding things like that Mozart is "Rap [explicit]". I don't have the mental capacity for it. Just give me the dumb hierarchy with (sym)links. (And bind mounts.) (But I wouldn't oppose an OS trying it.) "Let's ditch all IPC" … buuuut adds IPC right back in. Okay :). I suppose it's "only message passing". (And I do think message passing is a great default, and what most things need.) "Native apps are bloated". The most bloated apps I have are all the Electron ones. I don't think most people are concluding this... I'd love a multi-item clipboard, but the devil is in the details: sensitive items and large transfers. > it's the pure text nature that bothers me. Instead of chaining CLI apps together with text streams we need something richer, like serialized object streams (think JSON but more efficient). You have to consider the bulk cases. But very much yes; I wish at least that Unix had settled on typed bytestreams, like MIME+stream, and some form of content negotiation. Unix scripts are rife with examples of `a | b` where a & b speak different formats and I feel like it took forever to get good tooling around strongly specified formats (instead of just bullshitting with awk/grep/cut/tr/etc.) with tools like jq, yq, xsv, etc. I just wish applications could just have a means to understand "oh, I'm getting this kind of data" or "I have no idea what this is, and I shouldn't just GIGO". > In our new OS every window is tab dockable. Or side dockable. Or something else Have you tried a tiling WM? If you design an OS ... make the FS Unicode only, and ban "\n" from filenames. Life will be simpler.
TrevorFSmith 2y ago
It's interesting to me that this article was written ~seven years ago and already feels thoroughly outdated. The core problem is that its basic premise is that we need to scratch-write an OS that is essentially the Alto WIMP (1972), a few BeOS features (1995), and questionable improvements like forcing apps through a messaging bus a la Smalltalk (1972) or CORBA (1991). It isn't that the individual ideas are bad, per se, but if you're going to suggest write an OS from scratch then it really should be significantly different and inherently impossible to make by tweaking an existing OS, of which there are so, so many. Just an unbelievable number of OS options.
- scroot 2y ago
  Are there really so many OS options these days? Everything is pretty much "a Unix" of some sort, or else Windows. There is a key difference between today and the period of the 80s/90s when we had true OS efflorescence. Back then, the real concern was compatibility of software and in particular file/data formats. These days we have so many data format standards and protocols that a lot of this interoperability is essentially system-agnostic. I think that's a huge advantage.
abcde777666 2y ago
It's easy to make something lean when you can start from scratch. Backwards compatibility and large user bases resistant to change are overlooked as primary causal factors of the modern state of affairs, imo.
- goodthenandnow 2y ago
  I understood the ideia and agree with it. Just want to point out that starting an OS from scratch, even if reusing an existing kernel, is a monstrous epic. I find it difficult that one would be able to just use existing kernels like Linux without some considerable changes to support a more secure userspace (which seems like a requirement in order to do wild new things).
scroot 2y ago
Every time I boot up Squeak, I think about what the desktop os could have been had all the millions of dollars and hours of work gone in that direction instead
nuudlman 2y ago
A lot of this seems like it's done by Arcan (https://arcan-fe.com https://arcan-fe.com)
joshmarinacci 2y ago
Original author here. I have no idea how my 7 year old post made it to the front page today, but I’m happy to say that the situation is even worse than it was back then. Desktop OSes are even more bloated and less under user control since then.
- __d 2y ago
  Was there ever any effort to address these issues that came of your original post?
- 6510 2y ago
  An improved rant would be nice. I feel the webpage doesn't make enough effort to store the many valuable thoughts and idea the many readers have while reading it. Long ago I had a site that was technically a forum but the front page was an article rather than a table and the sections pointed to sub forums that were articles as well. The later pointed at the usual forum tables. I used the discussion/comments to improve the articles but it should really have been a wiki format. The text reminded me of the way FTP clients pop up the default text editor. Such a boring technology, no one thought anything of it. Why isn't my mail client doing this? It doesn't seem hard to make web browsers do it either. Trying to do code highlighting in a <textarea> is possibly the worse technology we have. We should really have a dedicated client for forms as well which seems to logically fold out of a database file system? The text also reminded me of Picassa the photo organizer, as soon as I had everything scanned and organized in it it felt like I couldn't use the normal file system anymore. It was such a clumsy clunker by comparison. Also hilarious was having to install Thunar because Nautilus couldn't open my big folders. I was also reminded of textpad. While Atom couldn't open files larger than 2mb the old textpad on a pentium 4 opened the 1+ TB wikipedia xml dump in 0 seconds. Editing was not sluggish, replace macros were not sluggish and saving the document worked just fine. The thought stuck with me: How the f did it just do that??? I think the first release in 1992 probably explains it. Squeezing wine from a rock was still a thing back then.
zzo38computer 2y ago
I wrote a response but it is too long (11019 bytes is too long, apparently). (I have saved a copy on my computer, so that I may post it later (fully or partially).) But, I have commented about most of the article, and about my own ideas of operating system design (which is capability-based); there are some similarities, but many things are done differently. However, some of the functionality they describe would be possible with how my ideas are doing, too.
- joshmarinacci 2y ago
  I’d love to hear about it.
  - zzo38computer 2y ago
    Most of my notes about operating system design are in paper, and in disorganized files in computer (some of which have not been updated from my changed ideas), I have written some on Hacker News as well, such as: https://news.ycombinator.com/item?id=40651272 https://news.ycombinator.com/item?id=40651272 https://news.ycombinator.com/item?id=40676995 https://news.ycombinator.com/item?id=40676995 (I had also written some on Usenet, and also on IRC.) Perhaps I could set up a NNTP for discussion, that work can be made to write the specifications (which can be changed as the actual implementation is made, in case that reveals problems with the specifications), and to discuss implementations.
    - Aerbil313 2y ago
      Is it possible you send me an email?
      - zzo38computer 2y ago
        I could, but I would prefer not to. Public communication would be preferable, such as by using NNTP. (I may set up a NNTP for this; I have a NNTP server set up already and can easily add a newsgroup.)
    - 2y ago
gavinhoward 2y ago
This is almost making me want to try out my ideas. https://gavinhoward.com/2024/05/a-new-architecture-for-a-gui-framework/ https://gavinhoward.com/2024/05/a-new-architecture-for-a-gui... https://gavinhoward.com/2024/05/a-better-os-design/ https://gavinhoward.com/2024/05/a-better-os-design/
debo_ 2y ago
The author's most recent post on this topic is here (2022): https://joshondesign.com/2022/09/21/idealos_mark6 https://joshondesign.com/2022/09/21/idealos_mark6
- lproven 2y ago
  That's interesting -- thanks. Looks to me like he has some of the same ideas as I had in this talk: https://fosdem.org/2024/schedule/event/fosdem-2024-3095-one-way-forward-finding-a-path-to-what-comes-after-unix/ https://fosdem.org/2024/schedule/event/fosdem-2024-3095-one-... I turned it into an article series: https://www.theregister.com/Tag/One%20Way%20Forward/ https://www.theregister.com/Tag/One%20Way%20Forward/
joshmarinacci 2y ago
If you want to know what I’m currently working on, and that helps in my prototypes: check out https://docs.trunk.io/ https://docs.trunk.io/
eviks 2y ago
Very sad article, so much wasted productivity potential... And it's no better today, unfortunately > Why can't I have a file in two places at once on my filesystem? Hard links do that, but as usual the UI around this isn't great
- PaulDavisThe1st 2y ago
  Only on the same filesystem
  - eviks 2y ago
    you mean this? > two places at once on my filesystem
    - PaulDavisThe1st 2y ago
      Well, it is unclear. In a POSIX-y context "filesystem" can refer to (a) the entire permanent storage system (b) a particular technology for organizing bits on a storage system (c) a subset of (a) that has been mount(1)-ed somewhere within (a) I tend to use (c) most often. No idea what the author or you might do.
andsoitis 2y ago
> Twitter user ekse has graciously translated this blog to Chinese. How far we've come.
danybittel 2y ago
"Essentially every application on my computer is a silo." So true. And so is every company. They have very little incentives to make your data accessible. Or is it "their" data? You can also go "turtles all the way down"... and discover: the object oriented programming model. It also hides, eh "encapsulates" your data, you only get access through messages. This is what operating systems are build on. And it works very well for Microsoft / Apple / Google. I think Unix had the right idea. Make the data part of the outside and programs only process this data. Programs should be stateless. Unfortunately Unix kind of got stuck in the text based world. Text is really great, because it's so visible. But there is no reason binary data can't be the same. You'd need to develop general purpose binary file format. That can be opened and edited outside of any application. Which would let you extract for example a single image of a video to edit in a paint app. Then make all of this live update, so that the mouse for example is just a struct with position and other data you can use everywhere. Then let us have history on all the data.
scroot 2y ago
Just think about the new kind of OS one could make when the computing system is built entirely of low-level actors. uFork [1] is a promising example, and has clear motivations [2]. [1] https://github.com/organix/uFork https://github.com/organix/uFork [2] http://www.dalnefre.com/wp/2022/08/memory-safety-simplifies-microprocessor-design/ http://www.dalnefre.com/wp/2022/08/memory-safety-simplifies-...
renox 2y ago
It's funny this article is technically correct but also totally wrong: if you can't run the application(web browser, game, office suite) you want on an OS, it maybe the 'Ideal' OS you're still going back to Windows..
- zzo38computer 2y ago
  There are ways around this in some cases: use emulation, modify the existing program to port it to the new system (if the source code is available), or write a new one. An ideal system will allow all three possibilities; although, which are possible also depends on the specific program that you want to run.

83 points by mmphosis 2y ago

HKH2 2y ago

Why not just build your OS on top of Linux? Then you can get driver support at least.

goodthenandnow 2y ago
Because Linux, the kernel, as well as other Unixes, don't have the right abstractions to easily build these. And there's all the old cruft. Just look at what Android had to do to implement a secure platform with somewhat of a well defined non-1980's OS API - it basically removed the Unix part of the OS and built a Microkernel-like thing on top of it. Driver support would be a very important part though.
- kimixa 2y ago
  > Just look at what Android had to do to implement a secure platform with somewhat of a well defined non-1980's OS API - it basically removed the Unix part of the OS and built a Microkernel-like thing on top of it. ...Huh? Pretty much all Android adds to the kernel is Binder, which is really a performance optimization for it's version of IPC, which today has equivalents that mean it probably wouldn't need it's own version now if it didn't need compatibility. The security model allows native apps, which have access to all the normal libc, or even direct syscalls. Some hardware access is marshalled through userspace daemons, which may implement security for things like camera/microphone access, but that's been a pretty common thing on other unix-style OSs for ages. Though the separation of each app into it's own UID might not have been the original intent, but it seems to work implemented with those same unix capabilities. Yeah, most apps probably don't use POSIX APIs directly, and instead the java Android APIs, but they are very much build on top of the "crufty 1980s OS APIs", rather than bypassing them.
  - goodthenandnow 2y ago
    Android blocks most of Linux syscalls with seccomp. They make heavy usage of SELinux. Binder is not just a simple IPC system. It requires userspace parts which aren't even the parties who are talking through IPC. Like you said it has lots of performance optimizations, and that requires it hooking up to a number of kernel subsystems. At the beginning it was met with a lot of resistance from kernel maintarners when tried to be merged. Why did Google came up with it? When talking about the 'Android sandbox' people usually talk about how Android gives each app an different uid/gid. But that's just the beginning of the story. It wasn't enough for sandboxing needs since the beginning. Thats why they took the binder idea from BeOS and used on Android. Since then Google has blocked everything. Native code is needed on any practical system, but today's native code has access mostly to nothing, just the common needs to run (like libc which is used due backward compatibility). The proper way to get something is through Binder (and even that is fine-grained controlled on a per process basis). Linux and Unixes in general don't implement true capabilities. Android does through Binder, but since Binder lives “inside Linux” everything else has to be blocked to make these capabilities useful. > Some hardware access is marshalled through userspace daemons Most of stuff are, at least initially, orchestrated through userspace software - daemons like you said, but mostly HALs (which are also daemons). The main takeaway is that binder is _the core_ of Android userspace. It's tightly integrated with SELinux and everything passes through it at some point. Permissions are enforced through it. In the end, the whole “binder subsystem” is architectured like a Microkernel system.
    - kimixa 2y ago
      I'd argue that permissions aren't really enforced by binder, just binder is used to talk to the endpoints that enforce permissions. Binder has no interaction with SELinux outside the policy requirements to allow apps to open it's device nodes - there's no interaction in the kernel code at all [0]. It's just an opaque data blob IPC mechanism, not tagging payloads with capabilities or similar in-kernel. At the point that binder was created, there wasn't a good kernel-level IPC system integrated into linux, true, but I'm not sure I agree with the idea that it's in any way tied into the capabilities/permissions system outside from being a communication channel to the userspace daemons things that do. And native code does have access to every service on the device - just as any app. It's just more painful than the Java layer, and some things aren't guaranteed to be stable over API versions. Less useful for apps, but still not part of the security model. At an extreme level, the java code is run in the same memory address space as any native code, there's no protection from that. Hell, some apps even used to patch dalvik/ART live. It was insane and fragile, but entirely possible. And the "disallowed" syscalls are those not exposed by libc though some more are restricted for user apps (mostly around UID management, which is the big "major" deviation from standard unix, and system stuff like loading kernel modules or rebooting...) [1] So maybe it doesn't allow some linux extensions, or could remove the "old" versions of replaced syscalls (when arguments were found not to be large enough, flags arguments added etc.). But there's not really anything missing from there as used in 99% of unix userspace programs anyway. So I maintain it's still a system build on top of a unix kernel and libc with a few additions on the side, and all that is still accessible to apps. Perhaps you think I'm including all the stuff people associate with modern "Desktop Linux"? Like X11 and/or wayland? Or the entire FHS? That's the main thing Android has completely divorced from, but that also isn't "Unix", or even really from the 80s even if it has some ancestors there. [0] https://android.googlesource.com/kernel/common/+/refs/heads/android-mainline/drivers/android/ https://android.googlesource.com/kernel/common/+/refs/heads/... [1] See SECCOMP*.txt and SYSCALLS.txt from https://android.googlesource.com/platform/bionic/+/refs/heads/main/libc/ https://android.googlesource.com/platform/bionic/+/refs/head...
- theamk 2y ago
  Eh the system described in the post is mostly about taking away (no filesystem access, no multiple IPC methods, no X11...) All new required software seems to be only userspace: - New IPC system: presumably it's abstracted, so can use any existing IPC for implementation. Or, purely as optimization, use Android's Binder or bring back bus1 (it was such a great idea) - Document Database: pure userspace; in the most extreme case use raw block device - Compositor / WM / Windows: this was always userspace already - Rebuild the apps: userspace as well Then once you new fancy APIs are ready, you use seccomp and/or per-app users and/or namespaces to isolate apps from everything except the few things people want.
  - goodthenandnow 2y ago
    > the system described in the post is mostly about taking away I agree. The hard part is, as usual, on the details. How do you take away stuff so that you still end up with an usable general purpose OS? That's definitely not easy, specially if you want to make it in a proper way, without hacking things up. For example, suppose you make the decision to use the Linux kernel, which is a good one - you get driver support, networking, filesystems, and all the good stuff and general primitives for a common OS of today.. Given that what one wants is to remove stuff, in order to make that secure (which is requirement) you'd need to isolate and block everything because on Linux everything is accessible to a process by default, and the common primitives for control are very basic - you'd have to do a _lot_ of work in userspace to block it. It would contaminate your userspace architecture with details and stuff. And then you'd probably end up needing something to make the userspace parts able to communicate with each other, like Binder on Androids case (see my other comment in this thread).

aredox 2y ago

The state of desktop paradigms has been one of great stagnation, if not outright regression. The VR fad is never going anywhere: its problems are an order of magnitude worse and engineers and companies nowadays seem unable to do basic 2D interface right.

m463 2y ago
I really notice this when I play PC games. They are easy to get into, they are highly customizable, and they teach you bit by bit until you're pretty powerful at doing things.
- paulryanrogers 2y ago
  Each game though is super narrow, and is built on OS primitives already established. An OS must be all things to all users and all devs, so some complexity and bloat are inevitable. Users don't want to start from scratch to learn every program they need. Also, they often bounce off of games with controls outside their comfort zone.
linguae 2y ago
I agree. In my opinion, the problem is that Microsoft and Apple discovered more lucrative markets, making their desktops less important. For Apple, the iPhone’s revenue completely dwarfs the Mac’s, and even services bring in more revenue than the Mac. Microsoft still makes a lot of money selling Windows licenses to OEMs and corporations, but somehow Microsoft still feels compelled to extract even more money from Windows by placing ads in the OS and by selling subscription services. As a fan of desktop GUI software that conforms to well-developed platform UI guidelines such as the old Apple Human Interface Guidelines, I believe the rise of mobile computing and the Web hurt the desktop. Software companies ship cross-platform applications using least-common-denominator frameworks instead of writing apps with UIs that are tailor-made to each platform. While this saves development time, and while it’s popular to say “having an Electron app is better than having no app,” it makes the desktop experience less consistent and more difficult to use. In addition, the desktop UI has become more mobile-like even in ways that are inappropriate for desktop computing, such as the proliferation of hamburger windows, the tendency to hide settings (thus requiring additional clicks to perform operations that used to be done in one or two clicks), oversized bars, title bars being crammed with buttons (making it harder to move windows via dragging the title bar), excessive padding, etc. There is a difference between making things easier for users versus dumbing things down, but I feel that a lot of software designers these days don’t know the difference. I would like to see a return to UI consistency across applications, and I’d also like to see a desktop environment that caters to power users. A dream of mine is some type of “component-based” desktop operating system, where everything is an object that can be manipulated through either a GUI, the command line, or programmatically. The backend would be inspired by technologies like Smalltalk, GNU Emacs, Lisp machines, and Apple OpenDoc, but with modern considerations such as security and parallelism.

Qxliang 2y ago

Is getting rid of command line interfaces really a good idea?I don't think so,a lot of people work with and are fond of the CLI tool.

theendisney 2y ago
Depends entirely on what you replace it with. A fun thought, key bindings and task visualisations could mimic cli
al_borland 2y ago
If the CLI were to go away, the OS would still need to have rich APIs for automation. While it may not be something the average user ever uses or knows about, it’s vital for power users and the enterprise.
- jcelerier 2y ago
  And then you want these automation APIs to be accessible through a REPL for quick prototyping and...
- vmfunction 2y ago
  Either that or the OS need to provide an API to allow major languages to be able to interact with the system other than the GUI. sh can be an add on.
  - goodthenandnow 2y ago
    Not sure if this is what you meant, but I'd like to expand a little bit on it. - This API should be programming language agnostic (like, with a well defined ABI). Ideally an IPC based one, with a well defined wire format and protocols. - Not a singular API, but an “API system” with extensibility taken into account from the beginning (with things like versioning). It should be able to support arbitrary new API definitions so that other apps could consume, and also produce. - Such an API system should be used by the OS itself too, of course. So the OS would have the resulting benefits and the whole system would be more uniform.

janice1999 2y ago

This article predates Apple's visionOS. I wonder what the author's take on it would be.

joshmarinacci 2y ago
I love Apple’s hardware and the metaphors are well thought out, but I’m disappointed that one of the best design companies in the world couldn’t come up with better uses than photos and having a virtual Mac monitor.
- pjmlp 2y ago
  Because they didn't want it to be another MetaQuest or HoloLens, thus there isn't much left as use cases.

GalaxyNova 2y ago

[flagged]

linguae 2y ago

This is a nice document, and many of the suggestions dovetail with my own ideas for a component-based operating system, which I wrote about in 2020 (http://mmcthrow-musings.blogspot.com/2020/04/a-proposal-for-flexible-composable.html http://mmcthrow-musings.blogspot.com/2020/04/a-proposal-for-...) but haven’t had the time to work on. I wonder if there’s been any follow-up work on Ideal OS?

zzo38computer 2y ago
Some of my ideas, also. I agree with the four items listed in "Problems with Today's Desktop Environments and Applications". However, these are not the only issues. (I even thing the older Windows 3.1 and even older UI were well enough; you don't need such fancy graphics.) Your "Monolithic Applications versus Composable Tools" also is reasonable. What you list about Unix philsophy are the good ideas of Unix philsophy; it is too bad that many modern programs don't support it. I would allow command-line and GUI and easily interact between them. My idea involves capability-based security with proxy capabilities, and a program can send/receive/wait with capabilities it has access to; a message send/received consists of bytes and/or capabilities. (It is also similar than actor model in some ways.) This is what the kernel does (and programs can create their own capabilities called "proxy capabilities"); all I/O is required to use capabilities. (When a program starts, it must receive an initial message containing some capabilities, so that it can do I/O.) My idea also is to make the format which will be called "Common Data Format"; it can store documents, databases, and other stuff, and can be used in command-line and GUI. (It also involves some of the ideas of OpenDoc, and of TRON Application Databus, but is different from both of them.) There are also many other considerations: hypertext file system, window indicators, security, TRON character code, network transparency, emulation, etc. About "Not a New Operating System", my idea would be that the specification can be done and implementations of the individual parts can be done independently and interchangeable, and that an implementation can be made to run stand-alone or inside of another operating system (something that Inferno also does); this can avoid the problems of drivers at first, while when it is made stand-alone, drivers can be made for it in that way, too. However, I do not intend to include a web browser by default. (For making such a thing on Linux, I don't know exactly what capsicum blocks; capabilities are supposed to required for all I/O including accessing the current date/time. My system idea is similar like if you have only socketpair to create new channels of communication, and being able to send/receive messages that may contain SCM_RIGHTS, and the ability to wait for objects.)

deathanatos 2y ago

Part of me wonders how much of this is hardware vendors releasing unusable hardware. I don't know what the hangup on the RPi is (that seems odd, given how prevalent Linux on the RPi is) but that & the keyboard RGB LEDs seems like your bog standard "the hardware can't or if it can, the vendor didn't bother to document anything". The FOSS world has done an incredible amount of work reverse engineering over the years, but the behavior of HW companies are more to blame, and that's not a problem you'll solve with tech. But hardware vendors have for years, and still do, just say "our use case, or the highway, you don't really own the thing you're buying." And there's not much out there for one to "vote with one's wallet" on it. "FS can't store a file in two spots" … I want to accuse OP of being a Windows user, but they're obviously not [only]. So that's … an odd critique? I still think hierarchical+symlinks beats trying to tag stuff, and I've lived through enough decades of AI trying to tag stuff and deciding things like that Mozart is "Rap [explicit]". I don't have the mental capacity for it. Just give me the dumb hierarchy with (sym)links. (And bind mounts.) (But I wouldn't oppose an OS trying it.) "Let's ditch all IPC" … buuuut adds IPC right back in. Okay :). I suppose it's "only message passing". (And I do think message passing is a great default, and what most things need.) "Native apps are bloated". The most bloated apps I have are all the Electron ones. I don't think most people are concluding this... I'd love a multi-item clipboard, but the devil is in the details: sensitive items and large transfers. > it's the pure text nature that bothers me. Instead of chaining CLI apps together with text streams we need something richer, like serialized object streams (think JSON but more efficient). You have to consider the bulk cases. But very much yes; I wish at least that Unix had settled on typed bytestreams, like MIME+stream, and some form of content negotiation. Unix scripts are rife with examples of `a | b` where a & b speak different formats and I feel like it took forever to get good tooling around strongly specified formats (instead of just bullshitting with awk/grep/cut/tr/etc.) with tools like jq, yq, xsv, etc. I just wish applications could just have a means to understand "oh, I'm getting this kind of data" or "I have no idea what this is, and I shouldn't just GIGO". > In our new OS every window is tab dockable. Or side dockable. Or something else Have you tried a tiling WM? If you design an OS ... make the FS Unicode only, and ban "\n" from filenames. Life will be simpler.

TrevorFSmith 2y ago

It's interesting to me that this article was written ~seven years ago and already feels thoroughly outdated. The core problem is that its basic premise is that we need to scratch-write an OS that is essentially the Alto WIMP (1972), a few BeOS features (1995), and questionable improvements like forcing apps through a messaging bus a la Smalltalk (1972) or CORBA (1991). It isn't that the individual ideas are bad, per se, but if you're going to suggest write an OS from scratch then it really should be significantly different and inherently impossible to make by tweaking an existing OS, of which there are so, so many. Just an unbelievable number of OS options.

scroot 2y ago
Are there really so many OS options these days? Everything is pretty much "a Unix" of some sort, or else Windows. There is a key difference between today and the period of the 80s/90s when we had true OS efflorescence. Back then, the real concern was compatibility of software and in particular file/data formats. These days we have so many data format standards and protocols that a lot of this interoperability is essentially system-agnostic. I think that's a huge advantage.

abcde777666 2y ago

It's easy to make something lean when you can start from scratch. Backwards compatibility and large user bases resistant to change are overlooked as primary causal factors of the modern state of affairs, imo.

goodthenandnow 2y ago
I understood the ideia and agree with it. Just want to point out that starting an OS from scratch, even if reusing an existing kernel, is a monstrous epic. I find it difficult that one would be able to just use existing kernels like Linux without some considerable changes to support a more secure userspace (which seems like a requirement in order to do wild new things).

scroot 2y ago

Every time I boot up Squeak, I think about what the desktop os could have been had all the millions of dollars and hours of work gone in that direction instead

nuudlman 2y ago

A lot of this seems like it's done by Arcan (https://arcan-fe.com https://arcan-fe.com)

joshmarinacci 2y ago

Original author here. I have no idea how my 7 year old post made it to the front page today, but I’m happy to say that the situation is even worse than it was back then. Desktop OSes are even more bloated and less under user control since then.

__d 2y ago
Was there ever any effort to address these issues that came of your original post?
6510 2y ago
An improved rant would be nice. I feel the webpage doesn't make enough effort to store the many valuable thoughts and idea the many readers have while reading it. Long ago I had a site that was technically a forum but the front page was an article rather than a table and the sections pointed to sub forums that were articles as well. The later pointed at the usual forum tables. I used the discussion/comments to improve the articles but it should really have been a wiki format. The text reminded me of the way FTP clients pop up the default text editor. Such a boring technology, no one thought anything of it. Why isn't my mail client doing this? It doesn't seem hard to make web browsers do it either. Trying to do code highlighting in a <textarea> is possibly the worse technology we have. We should really have a dedicated client for forms as well which seems to logically fold out of a database file system? The text also reminded me of Picassa the photo organizer, as soon as I had everything scanned and organized in it it felt like I couldn't use the normal file system anymore. It was such a clumsy clunker by comparison. Also hilarious was having to install Thunar because Nautilus couldn't open my big folders. I was also reminded of textpad. While Atom couldn't open files larger than 2mb the old textpad on a pentium 4 opened the 1+ TB wikipedia xml dump in 0 seconds. Editing was not sluggish, replace macros were not sluggish and saving the document worked just fine. The thought stuck with me: How the f did it just do that??? I think the first release in 1992 probably explains it. Squeezing wine from a rock was still a thing back then.

zzo38computer 2y ago

I wrote a response but it is too long (11019 bytes is too long, apparently). (I have saved a copy on my computer, so that I may post it later (fully or partially).) But, I have commented about most of the article, and about my own ideas of operating system design (which is capability-based); there are some similarities, but many things are done differently. However, some of the functionality they describe would be possible with how my ideas are doing, too.

joshmarinacci 2y ago
I’d love to hear about it.
- zzo38computer 2y ago
  Most of my notes about operating system design are in paper, and in disorganized files in computer (some of which have not been updated from my changed ideas), I have written some on Hacker News as well, such as: https://news.ycombinator.com/item?id=40651272 https://news.ycombinator.com/item?id=40651272 https://news.ycombinator.com/item?id=40676995 https://news.ycombinator.com/item?id=40676995 (I had also written some on Usenet, and also on IRC.) Perhaps I could set up a NNTP for discussion, that work can be made to write the specifications (which can be changed as the actual implementation is made, in case that reveals problems with the specifications), and to discuss implementations.
  - Aerbil313 2y ago
    Is it possible you send me an email?
    - zzo38computer 2y ago
      I could, but I would prefer not to. Public communication would be preferable, such as by using NNTP. (I may set up a NNTP for this; I have a NNTP server set up already and can easily add a newsgroup.)
  - 2y ago

gavinhoward 2y ago

This is almost making me want to try out my ideas. https://gavinhoward.com/2024/05/a-new-architecture-for-a-gui-framework/ https://gavinhoward.com/2024/05/a-new-architecture-for-a-gui... https://gavinhoward.com/2024/05/a-better-os-design/ https://gavinhoward.com/2024/05/a-better-os-design/

debo_ 2y ago

The author's most recent post on this topic is here (2022): https://joshondesign.com/2022/09/21/idealos_mark6 https://joshondesign.com/2022/09/21/idealos_mark6

lproven 2y ago
That's interesting -- thanks. Looks to me like he has some of the same ideas as I had in this talk: https://fosdem.org/2024/schedule/event/fosdem-2024-3095-one-way-forward-finding-a-path-to-what-comes-after-unix/ https://fosdem.org/2024/schedule/event/fosdem-2024-3095-one-... I turned it into an article series: https://www.theregister.com/Tag/One%20Way%20Forward/ https://www.theregister.com/Tag/One%20Way%20Forward/

joshmarinacci 2y ago

If you want to know what I’m currently working on, and that helps in my prototypes: check out https://docs.trunk.io/ https://docs.trunk.io/

eviks 2y ago

Very sad article, so much wasted productivity potential... And it's no better today, unfortunately > Why can't I have a file in two places at once on my filesystem? Hard links do that, but as usual the UI around this isn't great

PaulDavisThe1st 2y ago
Only on the same filesystem
- eviks 2y ago
  you mean this? > two places at once on my filesystem
  - PaulDavisThe1st 2y ago
    Well, it is unclear. In a POSIX-y context "filesystem" can refer to (a) the entire permanent storage system (b) a particular technology for organizing bits on a storage system (c) a subset of (a) that has been mount(1)-ed somewhere within (a) I tend to use (c) most often. No idea what the author or you might do.

andsoitis 2y ago

> Twitter user ekse has graciously translated this blog to Chinese. How far we've come.

danybittel 2y ago

"Essentially every application on my computer is a silo." So true. And so is every company. They have very little incentives to make your data accessible. Or is it "their" data? You can also go "turtles all the way down"... and discover: the object oriented programming model. It also hides, eh "encapsulates" your data, you only get access through messages. This is what operating systems are build on. And it works very well for Microsoft / Apple / Google. I think Unix had the right idea. Make the data part of the outside and programs only process this data. Programs should be stateless. Unfortunately Unix kind of got stuck in the text based world. Text is really great, because it's so visible. But there is no reason binary data can't be the same. You'd need to develop general purpose binary file format. That can be opened and edited outside of any application. Which would let you extract for example a single image of a video to edit in a paint app. Then make all of this live update, so that the mouse for example is just a struct with position and other data you can use everywhere. Then let us have history on all the data.

scroot 2y ago

Just think about the new kind of OS one could make when the computing system is built entirely of low-level actors. uFork [1] is a promising example, and has clear motivations [2]. [1] https://github.com/organix/uFork https://github.com/organix/uFork [2] http://www.dalnefre.com/wp/2022/08/memory-safety-simplifies-microprocessor-design/ http://www.dalnefre.com/wp/2022/08/memory-safety-simplifies-...

renox 2y ago

It's funny this article is technically correct but also totally wrong: if you can't run the application(web browser, game, office suite) you want on an OS, it maybe the 'Ideal' OS you're still going back to Windows..

zzo38computer 2y ago
There are ways around this in some cases: use emulation, modify the existing program to port it to the new system (if the source code is available), or write a new one. An ideal system will allow all three possibilities; although, which are possible also depends on the specific program that you want to run.