6 ms·

Amoeba is a famous example of a distributed OS https://en.wikipedia.org/wiki/Amoeba_(operating_system) https://en.wikipedia.org/wiki/Amoeba_(operating_system)

by als0 4y ago

api 4y ago
It’s too bad this research didn’t continue. Instead we got a giant tower of kludges including containers, Kubernetes, service meshes, endless API design problems, endless reinventing of data serialization and protocols, and so on.
- deleted 4y ago
  [deleted]
- johnisgood 4y ago
  Or something like https://en.wikipedia.org/wiki/Sprite_(operating_system) https://en.wikipedia.org/wiki/Sprite_(operating_system) but with a microkernel!
- mananaysiempre 4y ago
  The academic environment where that research was the thing to do if you wanted to do systems is the academic environment that produced Pike’s “Systems Software Research Is Irrelevant”[1]. It seems to have succumbed to two relatively common afflictions, though I don’t know of another case of both simultaneously: - Inbreeding of ideas: researchers inventing and solving problems noöne actually has, without observing the discipline necessary to remain interesting and avoid emotional attachments (as is done in maths, with varying success). In the distributed OS case, the model in people’s heads (including Pike’s with Plan 9!) seems to have been a closed campus network, which is close to some of the areas people care now, but still isn’t quite any of them. The social problem of the system not being able to take over the world, thus necessitating explicitly specified protocols and loose coupling, also does not seem to have received much consideration. (Except for Microsoft, who has tried—deliberately or accidentally—to weaponize the homogeneity requirements of some of that research in building Windows domain networking.) Some of the kludges aren’t—they are solving social problems. - Being before its time: sounds glorious and heroic, in reality makes people cold, lonely, and more often than not broke. The most commonly known instances of this are for individuals (or other actors capable of having a coherent volition, such as companies), but the distributed OS story feels like this happened to the community as a whole. [1] http://doc.cat-v.org/bell_labs/utah2000/utah2000.html http://doc.cat-v.org/bell_labs/utah2000/utah2000.html
  - mike_hearn 4y ago
    Great document, thanks for the link. I think the main issue is actually more related to the trend in academia in which the 'least publishable unit' is ever more important. Pike's essay touches on this. Coming up with an interesting new OS design is hard, and implementing one is worse, so people don't try because it'd take too long and there wouldn't be enough papers in it for anyone. Companies still do OS research (see Singularity and Midori by Microsoft, Fuschia by Google) but don't publish much about it. The problem is to become sustainable you need users, and to get users you really have to be above the core OS these days because that's where you can focus on concepts instead of getting distracted by kernel and driver writing (not much interesting to do there). If you want to experiment with microkernels, do it as a collection of processes in the Linux userland and if you want to extend that to get users, make it run on Windows too. Chrome is in effect a kind of microkernel-ish operating system, seen from this perspective. It consists of a pile of sandboxed components connected by a sophisticated IPC system called Mojo. It also handles properly cases that microkernel research sometimes ignores or punts to the user, like how to gracefully handle if the other end of a connection crashes. I feel strongly that there's still lots of interesting research to do in the operating systems space but if I were to do it, I would not start by writing an actual OS. I'd start by exploring the ideas as an set of ordinary programs that could run on a regular OS. Maximize the window and start gradually eliminating dependencies on high level APIs provided by the host, if you really want to. Then you can explore concepts like single address space OS's, ultra-fine-grained microkernels, better shells, polyglot programming, distributed computing etc, without getting bogged down by irrelevant detail.
    - still_grokking 4y ago
      I think this misses the whole point. If you want novel computer systems you can't build them on top of the old ones. That's the whole point of Pike. An OS is the thing in about Linux (kernel) scope. The whole point is that this is something that is by no mean "done". If you want innovation in this space you need to build exactly such a thing from scratch! But the OS is only one part of systems design. Doing something novel in this area would be to continue where we stopped around 1970. There was no progress since than. Only one computer architecture with only one OS left. Since than nothing truly innovative was even tried. (OK, to be fair, almost nothing. There been things like Intel's iAPX 432, or Microsoft's Midori, or things like the Connection Machine. But mainstream is still on the std. tech form the 70'. Even mainframes are actually technological more advanced, as they had for example multiprocessing, VMs and containers already decades ago). I for my part think that if we would rethink computers form the ground up in light of today's possibilities they would be very very different. Alone the security hell today is only there because we still want to run "normal C programs". Everybody knows that this is not fixable without throwing away some of the old concepts. Still nobody wants to move. (Almost perfectly secure computers would be possible. You "just" need to build capability security into CPUs, the OS, and all apps…). Or another example: Scaling to many many cores would be much easier if computers would be fully data-flow aware. But currently we like to pretend that our CPUs are still PDP-7s. They need to offer the exact same interface (conceptually) or else this would not match the C abstract machine any more 1:1… But the CPU than needs to use a built in "hardware JIT" to translate the sequential command streams it receives to its internal data-flow operations. Somethings that's extremely inefficient and overly complicated—besides that it conceptually can't even help much. There is only so much parallelism extractable form an inherently sequential description of computation… (The most crazy part about this is that we have plenty information available on the high level which we happy erase during compilation, so we can present a "nice flat command stream" to the CPU. That's imho pure insanity). But nobody dares to rethink such systems as a whole: which means form the languages and their abstract machines, through the concrete hardware, the OS level, up to application design. Things like the von Neumann bottleneck were discussed already almost 50 years ago. The issue remains open to this day. Just put another layer of caches on top. Nobody cares about efficiency anyway… Because rewriting software would be to expensive. So the hardware can't be improved. So software needs to optimize for the old hardware concepts. Which leads to the dead-lock in which we're currently, where no substantial improvement can be made any more. We're in this trap since 50 years. And that's why systems research is absolutely dead by now. My 2ct. Sorry for the long post. But I have strong feelings about that topic.
- startupsfail 4y ago
  Apparently Python was developed for Amoeba. So in a way some strands of that research continued.
  - 1MachineElf 4y ago
    I wish there was a way to run Ameboa and early Python together on QEMU, but the HTTP download links on the Ameboa site are down.
    - mananaysiempre 4y ago
      Try the FTP archive at https://archive.org/details/2014.11.ftp.cs.vu.nl https://archive.org/details/2014.11.ftp.cs.vu.nl maybe, it seems to have the Amoeba files (unlike mirrorservice.org, which only has the Minix ones). The ArchiveOS page at https://archiveos.org/amoeba/ https://archiveos.org/amoeba/ linked from the Wikipedia page also has some.
_448 4y ago
Apache Mesos, though not a ground up OS, it does have the aim of sharing resources across clusters.
- zasdffaa 4y ago
  Apache Mesos is an OS??
  - nix23 4y ago
    https://mesos.apache.org/ https://mesos.apache.org/ >Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines
    - still_grokking 4y ago
      It's still not an OS in the classical sense. It's a regular user-mode executable. The idea is that it provides a kind of "virtual OS" to the applications build against it. But Mesos needs a regular OS as runtime on the nodes of the cluster.
      - skissane 4y ago
        > It's a regular user-mode executable. I don't think that necessarily excludes something from being an OS. Historically, a number of Unix ports ran as applications on top of a non-Unix. AT&T's early 1980s port of Unix to IBM S/370 mainframes, ran Unix as an application on top of IBM's TSS/370 operating system. [0] The Interdata 7/32 port done at Wollongong University in Australia, initially ran under Interdata's OS/32 operating system, although once the port had advanced sufficiently to run on bare metal, OS/32 was ripped out from under it. [1] The first versions of Cray's UNIX port, UNICOS, supported running as a guest under their earlier COS operating system (which was a batch processing operating system, modelled after that of CDC mainframes.) I still don't think Mesos counts though. Applications running under Mesos access resources by calling the Linux APIs; Mesos configures the Linux environment the application runs under, but at runtime it steps out of the way. If, hypothetically, Mesos had exposed its own API to applications running under it – separate from the Linux API, even if ultimately implemented using it – and forced them to use that rather than call the host OS APIs directly – then I would count it as an OS. But, that would be a rather different thing from what Mesos actually is. (And all those points apply to Kubernetes too.) With talk about WebAssembly as a container format – if you had something like Mesos/Auroroa/K8S/etc which ran WebAssembly containers only, so that applications running under it could only access WASI and other WebAssembly-based APIs – that would be closer to a true OS, even if it happened to run on top of the Linux kernel. You could swap out the Linux kernel for another – BSD, Windows, Illumos, XNU, GNU Hurd, L4, Fuschia, z/OS, whatever – and the apps shouldn't be able to tell the difference. [0] https://ieeexplore.ieee.org/document/6771917 https://ieeexplore.ieee.org/document/6771917 [1] http://bitsavers.informatik.uni-stuttgart.de/bits/Interdata/32bit/unix/univWollongong_v6/miller.pdf http://bitsavers.informatik.uni-stuttgart.de/bits/Interdata/...
forgotpwd16 4y ago
Moreover it is also microkernel-based.

Amoeba is a famous example of a distributed OS https://en.wikipedia.org/wiki/Amoeba_(operating_system) https://en.wikipedia.org/wiki/Amoeba_(operating_system)

by als0 4y ago

Amoeba is a famous example of a distributed OS https://en.wikipedia.org/wiki/Amoeba_(operating_system) https://en.wikipedia.org/wiki/Amoeba_(operating_system)

api 4y ago

It’s too bad this research didn’t continue. Instead we got a giant tower of kludges including containers, Kubernetes, service meshes, endless API design problems, endless reinventing of data serialization and protocols, and so on.

deleted 4y ago
[deleted]
johnisgood 4y ago
Or something like https://en.wikipedia.org/wiki/Sprite_(operating_system) https://en.wikipedia.org/wiki/Sprite_(operating_system) but with a microkernel!
mananaysiempre 4y ago
The academic environment where that research was the thing to do if you wanted to do systems is the academic environment that produced Pike’s “Systems Software Research Is Irrelevant”[1]. It seems to have succumbed to two relatively common afflictions, though I don’t know of another case of both simultaneously: - Inbreeding of ideas: researchers inventing and solving problems noöne actually has, without observing the discipline necessary to remain interesting and avoid emotional attachments (as is done in maths, with varying success). In the distributed OS case, the model in people’s heads (including Pike’s with Plan 9!) seems to have been a closed campus network, which is close to some of the areas people care now, but still isn’t quite any of them. The social problem of the system not being able to take over the world, thus necessitating explicitly specified protocols and loose coupling, also does not seem to have received much consideration. (Except for Microsoft, who has tried—deliberately or accidentally—to weaponize the homogeneity requirements of some of that research in building Windows domain networking.) Some of the kludges aren’t—they are solving social problems. - Being before its time: sounds glorious and heroic, in reality makes people cold, lonely, and more often than not broke. The most commonly known instances of this are for individuals (or other actors capable of having a coherent volition, such as companies), but the distributed OS story feels like this happened to the community as a whole. [1] http://doc.cat-v.org/bell_labs/utah2000/utah2000.html http://doc.cat-v.org/bell_labs/utah2000/utah2000.html
- mike_hearn 4y ago
  Great document, thanks for the link. I think the main issue is actually more related to the trend in academia in which the 'least publishable unit' is ever more important. Pike's essay touches on this. Coming up with an interesting new OS design is hard, and implementing one is worse, so people don't try because it'd take too long and there wouldn't be enough papers in it for anyone. Companies still do OS research (see Singularity and Midori by Microsoft, Fuschia by Google) but don't publish much about it. The problem is to become sustainable you need users, and to get users you really have to be above the core OS these days because that's where you can focus on concepts instead of getting distracted by kernel and driver writing (not much interesting to do there). If you want to experiment with microkernels, do it as a collection of processes in the Linux userland and if you want to extend that to get users, make it run on Windows too. Chrome is in effect a kind of microkernel-ish operating system, seen from this perspective. It consists of a pile of sandboxed components connected by a sophisticated IPC system called Mojo. It also handles properly cases that microkernel research sometimes ignores or punts to the user, like how to gracefully handle if the other end of a connection crashes. I feel strongly that there's still lots of interesting research to do in the operating systems space but if I were to do it, I would not start by writing an actual OS. I'd start by exploring the ideas as an set of ordinary programs that could run on a regular OS. Maximize the window and start gradually eliminating dependencies on high level APIs provided by the host, if you really want to. Then you can explore concepts like single address space OS's, ultra-fine-grained microkernels, better shells, polyglot programming, distributed computing etc, without getting bogged down by irrelevant detail.
  - still_grokking 4y ago
    I think this misses the whole point. If you want novel computer systems you can't build them on top of the old ones. That's the whole point of Pike. An OS is the thing in about Linux (kernel) scope. The whole point is that this is something that is by no mean "done". If you want innovation in this space you need to build exactly such a thing from scratch! But the OS is only one part of systems design. Doing something novel in this area would be to continue where we stopped around 1970. There was no progress since than. Only one computer architecture with only one OS left. Since than nothing truly innovative was even tried. (OK, to be fair, almost nothing. There been things like Intel's iAPX 432, or Microsoft's Midori, or things like the Connection Machine. But mainstream is still on the std. tech form the 70'. Even mainframes are actually technological more advanced, as they had for example multiprocessing, VMs and containers already decades ago). I for my part think that if we would rethink computers form the ground up in light of today's possibilities they would be very very different. Alone the security hell today is only there because we still want to run "normal C programs". Everybody knows that this is not fixable without throwing away some of the old concepts. Still nobody wants to move. (Almost perfectly secure computers would be possible. You "just" need to build capability security into CPUs, the OS, and all apps…). Or another example: Scaling to many many cores would be much easier if computers would be fully data-flow aware. But currently we like to pretend that our CPUs are still PDP-7s. They need to offer the exact same interface (conceptually) or else this would not match the C abstract machine any more 1:1… But the CPU than needs to use a built in "hardware JIT" to translate the sequential command streams it receives to its internal data-flow operations. Somethings that's extremely inefficient and overly complicated—besides that it conceptually can't even help much. There is only so much parallelism extractable form an inherently sequential description of computation… (The most crazy part about this is that we have plenty information available on the high level which we happy erase during compilation, so we can present a "nice flat command stream" to the CPU. That's imho pure insanity). But nobody dares to rethink such systems as a whole: which means form the languages and their abstract machines, through the concrete hardware, the OS level, up to application design. Things like the von Neumann bottleneck were discussed already almost 50 years ago. The issue remains open to this day. Just put another layer of caches on top. Nobody cares about efficiency anyway… Because rewriting software would be to expensive. So the hardware can't be improved. So software needs to optimize for the old hardware concepts. Which leads to the dead-lock in which we're currently, where no substantial improvement can be made any more. We're in this trap since 50 years. And that's why systems research is absolutely dead by now. My 2ct. Sorry for the long post. But I have strong feelings about that topic.
startupsfail 4y ago
Apparently Python was developed for Amoeba. So in a way some strands of that research continued.
- 1MachineElf 4y ago
  I wish there was a way to run Ameboa and early Python together on QEMU, but the HTTP download links on the Ameboa site are down.
  - mananaysiempre 4y ago
    Try the FTP archive at https://archive.org/details/2014.11.ftp.cs.vu.nl https://archive.org/details/2014.11.ftp.cs.vu.nl maybe, it seems to have the Amoeba files (unlike mirrorservice.org, which only has the Minix ones). The ArchiveOS page at https://archiveos.org/amoeba/ https://archiveos.org/amoeba/ linked from the Wikipedia page also has some.

_448 4y ago

Apache Mesos, though not a ground up OS, it does have the aim of sharing resources across clusters.

zasdffaa 4y ago
Apache Mesos is an OS??
- nix23 4y ago
  https://mesos.apache.org/ https://mesos.apache.org/ >Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines
  - still_grokking 4y ago
    It's still not an OS in the classical sense. It's a regular user-mode executable. The idea is that it provides a kind of "virtual OS" to the applications build against it. But Mesos needs a regular OS as runtime on the nodes of the cluster.
    - skissane 4y ago
      > It's a regular user-mode executable. I don't think that necessarily excludes something from being an OS. Historically, a number of Unix ports ran as applications on top of a non-Unix. AT&T's early 1980s port of Unix to IBM S/370 mainframes, ran Unix as an application on top of IBM's TSS/370 operating system. [0] The Interdata 7/32 port done at Wollongong University in Australia, initially ran under Interdata's OS/32 operating system, although once the port had advanced sufficiently to run on bare metal, OS/32 was ripped out from under it. [1] The first versions of Cray's UNIX port, UNICOS, supported running as a guest under their earlier COS operating system (which was a batch processing operating system, modelled after that of CDC mainframes.) I still don't think Mesos counts though. Applications running under Mesos access resources by calling the Linux APIs; Mesos configures the Linux environment the application runs under, but at runtime it steps out of the way. If, hypothetically, Mesos had exposed its own API to applications running under it – separate from the Linux API, even if ultimately implemented using it – and forced them to use that rather than call the host OS APIs directly – then I would count it as an OS. But, that would be a rather different thing from what Mesos actually is. (And all those points apply to Kubernetes too.) With talk about WebAssembly as a container format – if you had something like Mesos/Auroroa/K8S/etc which ran WebAssembly containers only, so that applications running under it could only access WASI and other WebAssembly-based APIs – that would be closer to a true OS, even if it happened to run on top of the Linux kernel. You could swap out the Linux kernel for another – BSD, Windows, Illumos, XNU, GNU Hurd, L4, Fuschia, z/OS, whatever – and the apps shouldn't be able to tell the difference. [0] https://ieeexplore.ieee.org/document/6771917 https://ieeexplore.ieee.org/document/6771917 [1] http://bitsavers.informatik.uni-stuttgart.de/bits/Interdata/32bit/unix/univWollongong_v6/miller.pdf http://bitsavers.informatik.uni-stuttgart.de/bits/Interdata/...

forgotpwd16 4y ago

Moreover it is also microkernel-based.