10 ms·

The End of the General Purpose Operating System

180 points by grey-area 10y ago

Animats 10y ago
This makes sense. The server OS underneath ought to be a lot smaller than Linux or Windows. No interactive logon support, no display support, no printer support, no hot plugging support, few drivers, and no battery management. And, most importantly, few changes. The OS underneath ought to be simple enough to be installed for the life of the hardware. The real "operating system" is the container orchestration system.
- noir_lord 10y ago
  Then you put all your dependencies on top and realise how large that is and that by bundling their dependencies with those dependencies security becomes a problem keeping everything up to date so someone comes up with the idea of sharing some of those dependencies and since they are common across multiple things they need a snappy name...hmm, sharing and re-usable how about..libraries, shared libraries in fact. I jest but as an industry we do seem have an almost metronome-like tendency to lurch from one thing and back.
  - walterbell 10y ago
    Lurching is a proven technique for displacement of incumbents.
  - pjmlp 10y ago
    You can share libraries without having an underlying OS, a rich runtime playing OS role is enough. For example, all JEE containers allow for sharing libraries among applications, and they can be running bare metal for what I care. I can even control in which order they get loaded in each application.
    - deleted 10y ago
      [deleted]
    - MrBuddyCasino 10y ago
      Oh the horror. Thank god we don't do that anymore.
      - eveningcoffee 10y ago
        Unfortunately, or fortunately this is still what we do. Even on the web scale.
- deleted 10y ago
  [deleted]
- Hello71 10y ago
  > No interactive logon support technically doesn't come by default with Linux (the kernel) anyways > no display support can be configured out. I'm pretty sure you can even configure out the whole TTY subsystem if you want, so you can't even use a serial port to debug your madness. > no printer support who still uses parallel ports? for anything else (except usblp which basically never works) you need CUPS anyways > no hot plugging support last I checked, hotplug is mandatory on x86 (but only for certain components, so you could configure out say USB support if you wanted) > few drivers sure, you can compile your own kernel if you want > no battery management I don't really know what "battery management" means. > The OS underneath ought to be simple enough to be installed for the life of the hardware. you can already try this with Linux. then you run in to problems like "what happens when there's a vulnerability in Xen", because https://marc.info/?l=openbsd-misc&m=119318909016582 https://marc.info/?l=openbsd-misc&m=119318909016582.
  - microcolonel 10y ago
    RISC-V's privilege separation model is promising for finally supporting hypervisors which probably work.
  - johncolanduoni 10y ago
    > I don't really know what "battery management" means I assume he means "power management" (i.e. ACPI on x86). You can run the system without taking control of the ACPI hardware, but this is a terrible idea. It'll affect your ability to run the CPU properly, even assuming you don't care about power usage (and large server deployments sure do!). Not to mention all the nasty bugs and problems you'll run into because the hardware wasn't designed to run for long periods of time without ACPI active.
    - asadjb 10y ago
      But would that matter, given more and more of our machines are running on a hypervisor of some sort? I know it's almost mandatory for the base machine running on actual hardware, but given that most of the servers we use now are just virtualized, how bad would removing ACPI support be? Also, would it be useful to remove that?
- gigatexal 10y ago
  Compile a custom kernel with only the hardware in the server and then only install packages you need. Boom done.
- oso2k 10y ago
  This was called DOS but was deemed not sufficient for people who wanted multi-tasking.
- js8 10y ago
  This was already invented, it's called z/VM. You have CP, which is a hypervisor (for any operating system), and CMS, which is a single-user OS that handles talking to the user.
- gaius 10y ago
  This was called Mach, check it out https://en.wikipedia.org/wiki/Mach_%28kernel%29 https://en.wikipedia.org/wiki/Mach_%28kernel%29
walterbell 10y ago
There is a proliferation of OS-native hypervisors (Windows 10, macOS, FreeBSD, OpenBSD, Linux) which use hardware virtualization to isolate workloads which may be VMs, containers, unikernels, apps or even processes.
dkarapetyan 10y ago
Wise man once said the only thing new is the history you don't already know. The process is a perfectly good unit of distribution, does not require orchestration, integrates nicely with PID1, leverages OS services for getting work done, can talk over the network, start other processes, etc. The reason containers have taken a foothold is not because the process is not a good unit of abstraction and distribution. The reason is dependencies have gotten out of hand. Ask anyone that deploys fat jars to run on the JVM. They've been leveraging containers for ages now. Ask the golang folks how they like deploying single binaries. The OS is not going away. Better process isolation is the future so I'm gonna say the author has a slightly too futuristic view of things. The current container ecosystem and the churn around it is a half-measure for better process management.
- xenadu02 10y ago
  The history of virtualization has shown us that it is far easier to containerize existing software than rewrite it to support strict isolation. Most OSes (sans macOS) aren't really setup for sandbox-by-default either. The mainframe industry already went through this years ago. They invented virtualization and leveraged it to transition through major new CPU architectures, complete re-designs of IO stacks, and switching out entire operating systems. None of these are new arguments. Virtualization won then and it will win again. You could say the entire history of the industry is increasing levels of abstraction; containers and virtualization are just another step along that path.
- pjmlp 10y ago
  The day I moved from C++ back to Java and other languages with richer runtimes at work, was when I stopped caring about the underlying OS. A JEE container, is already a full OS adding the remaining services that a plain JSE installation might still lack, it doesn't need yet another layer to waste resources and add more administration work in maintenance and security.
  - sqeaky 10y ago
    I write in C++ and I do not care about the underlying 98% of the time anyway. That 2% of the time I might write something that needs a kernel call, but I will make a class or some other abstraction that represents it and put all my different implementations in there. For code that cares about word size or something similarly pervasive but platform dependent there are templates and constepxr to have the compiler evaluate things. I won't be putting constants like 32 or 64 in my I will do things like "align_to(system_details::cache_line_size)" and let my compiler handle the details.
hannesm 10y ago
Processes? Seriously if there's only a single one, why would you waste resources on process management and process information? Just get over it ;) http://unikernel.org http://unikernel.org https://mirage.io https://mirage.io :D
- icebraining 10y ago
  There isn't a single one, there's at least one per container.
pjc50 10y ago
This seems like another point on the divergence from the traditional security model. In the 70s, the software on a computer was entirely controlled by the system administrator; the software was presumed secure and the threat was from the users. Users needed to be partitioned. In the present day, there's only one user who is also the system administrator, but the software is the threat to itself and others. True for both server-containers and mobile OSs.
- digi_owl 10y ago
  I dunno, seems more like the issue is that the outside attacker's path has gone from "dumb" to "smart". Where before access was done via a limited terminal protocol, these days it is done via a much richer protocol (or stack of protocols, what with tcp carrying http carrying json or some such). I don't think it was ever that local software was presumed secure. After all you wanted as little as possible to be suid, and if it was it should be as small a piece of code as possible to limit the chance of exploitable bugs.
- marssaxman 10y ago
  Yes! This is an argument I've been making for several years now: the user-centric security model is obsolete and unhelpful, because most computers have either one or zero users in the traditional sense. The whole unix-style reduced-privileges-plus-sudo approach that Windows and MacOS have copied is a nuisance which doesn't really solve the problem; the permissions systems in Android and iOS are a little closer to the mark. Qubes is a good step forward. The real problem is exactly what you said: we can't fully trust software, even software which is not explicitly malicious, because software can be exploited, and because software authors sometimes want to be "helpful" in ways we'd really rather they weren't. Every piece of software should run within a sandbox, and the human user should have complete control over which resources are or are not exposed to each sandbox; that's the future operating system I want to see. I did some exploration around the idea of doing this with hypervisors and unikernels (http://www.github.com/marssaxman/fleet http://www.github.com/marssaxman/fleet) but it got to look too much like rewriting all the software in the world. Containers are less elegant, but seem to be a more practical way of moving in the right direction.
digi_owl 10y ago
The title and the content are wildly divergent...
cyphar 10y ago
I don't really agree (I'm one of the developers of ocid / cri-o). Even with a container-manager-as-init you still need to administer your control plane (updates, configuration, so on). Now, you could go the CoreOS way of creating an "administration 'container'" which bindmounts / to somewhere inside the container -- that works but I would still consider it to not be any different from ssh-ing into the machine as root (it removes the need for sshd and means that you can manage the server through the same tooling you manage containers with, but the purpose is the same -- give you a general purpose OS environment you can use to administer things). Not to mention that your actual containers will still contain general purpose OS images. Also, as an aside, cri-o would be a very bad choice for PID 1. It's specifically designed so that you don't need it to be a long-running daemon (which is something that Docker cannot do and is actually a very useful feature that we hope to keep). And you can't have your PID 1 just exit whenever it wants to.
- justincormack 10y ago
  Even rancher no longer uses container manager as pid 1; these systems being discussed are minimal not single binary. (Only systemd is putting some "container management" features into pid 1).
  - icebraining 10y ago
    Which container management features is systemd putting into pid 1?
    - cyphar 10y ago
      machinectl and systemd-nspawn. Namely, systemd now manages container processes if you ask it to. My experience with the systemd cgroup handling doesn't fill me with much faith on this topic.
      - icebraining 10y ago
        systemd does manage containers, but from what I can tell, that functionality is not in PID 1, is it?
hoodoof 10y ago
Am I the only person who thinks that Docker adds unneeded complexity?
- gerbilly 10y ago
  No, you are not the only one :-)
- paulddraper 10y ago
  The world has 7 billion people. Very unlikely none share a certain idea, particularly that one.
- runeks 10y ago
  How do you think Docker adds unneeded complexity? As far as I can see it simplifies a non-trivial task (automated deployment). A while ago I discovered that I can build my app (Haskell) as a Docker image, using the "stack" build tool. As far as I can see, Docker adds value by providing a layer of abstraction that makes building deployment images as easy as building the executable.
tony-allan 10y ago
There is ample evidence that we cannot write software that can be installed, run and updated in a simple and secure manner. Software is too complex with too many dependencies. Hackers are are better than us almost every time -- and once in the can bounce around at will. I just want to run my own containers or those written by others on commodity services from Amazon, Google or Microsoft. Or even spin up my own environment if it suits me. I want to be able to install and remove software quickly, completely and whenever I want. I especially want to be able to install software I don't fully trust with no fear of consequences. I want to be able to install multiple versions and even multiple instances. I want my PC (Mac/Linux/Windows) to be a thin hypervisor with everything in a container, including IO and UI. I still want open source projects on GitHub that I can fork and contribute changes to at will. I want containers to be small and ultra simple. All of this means that my "OS" only needs to manage containers and resources. All traditional elements sould be in their own container such as logging, authentication, window managers.
- taeric 10y ago
  You seem to be implying that containers are somehow immune to hackers... Truth is, it is relatively easy to keep a machine secure. Just not if you want it to be a convenient machine. You think you want containers all isolated from each other. Except for the uncountable times and ways you want to exchange data between them. This, at the least, includes the data of who you are, as the user.
  - johncolanduoni 10y ago
    > Truth is, it is relatively easy to keep a machine secure. Just not if you want it to be a convenient machine. So the people who build high assurance systems (which currently require a ton of work) are all too focused on convenience?
    - taeric 10y ago
      Getting work done is a huge convenience. :) Specifically, remote access is a huge convenience that by itself moves the problem out of the easy territory.
winter_blue 10y ago
Library operating systems like IncludeOS are a further extension of this phenomenon: http://www.includeos.org http://www.includeos.org With IncludeOS, you don't even have a full kernel. You just have the bare necessities, and run your code in real mode in ring 0.
- cheiVia0 10y ago
  So your web server code has direct write access to the hard drive firmware?
  - yellowapple 10y ago
    If you run it on bare metal. I reckon the point of IncludeOS is to run an application directly on a hypervisor like Xen (instead of needing a separate guest OS).
noescape 10y ago
It's happening for the operating system. Will it happen for the programming language?
erikpukinskis 10y ago
Ah, the grand cycle of layering and integration. You build a machine that does a thing, modify it to do another, and another, and another, until eventually someone says We could have a layer that did all of this! The layer is introduced, applications are rewritten to target the layer, and people slowly lose touch with what the world looks like beneath The Interface. And some things that should not have been forgotten were lost. History became legend. Legend became myth. As people target The Interface, it grows into a more and more general purpose machine. Layers of indirection build up. Once simple tasks must propagate up and down a big stack. Until one day, someone stumbles across the layer beneath. Gosh, the underlying system does 99% of what we need, out of the box. Why do we need this layer at all? We can do most of what we need with a couple single purpose tools. And the rest of the complexity can be taken up by the application layer. I don't mind doing a little more configuration there if I can get a huge performance and complexity win. And the developers, frustrated with how big and bloated their layers have been feeling, flock to this new simple tool, and they port their applications, with a little extra boilerplate and big complexity wins. And then they port another, and another. Until someone realizes We could have a layer that did all of this! And it might seem pointless, when I write it up in this snarky way, but it absolutely is not. This is the process by which we discover the fundamental building blocks of software. Each time we add a layer, and each time we take one away, we learn something new about what information is. I love it.
- dfischer 10y ago
  Well damn said sir. I was waiting for a sarcastic jaded snarky ending but you turned it into the essence of technology. Kudos. This is exactly it. :)
- mycall 10y ago
  All that complexity is entropy, information's villain.
- deleted 10y ago
  [deleted]
- candiodari 10y ago
  I don't think that's really the issue with Android/IOS operating systems. The issue is not that they can't be general purpose operating systems, it's that they aren't used that way. People seem to want applications that do one or two things at most and cost 1$ that they have to re-buy every 2 months. That does not result in people writing software that is very useful for a large part of a particular field, but takes 2-3 years to write. But the fault is in the app store and the lemon market it creates, not in the operating system.
theamk 10y ago
I feel that author is very enthusiastic, but there are some pretty serious errors: "... fight between Docker and systemd is inevitable" -- this fight has been going for a while (at least a year): https://lwn.net/Articles/676831/ https://lwn.net/Articles/676831/ "... The reason why you'll do this, rather than compose everything yourself, is compatability. Whether it's kernel versions, file system drivers, operating system variants or a hundred variations that make your OS build different from mine. Building and testing software that runs everywhere is a sisyphean task. " -- kernel versions, file system drivers, operating system variants (in form of docker daemon version) are still going to be around and would still affect containers. So you traded "test on Fedora, RHEL, Ubuntu" with "test on AWS docker, Tectonic, OpenShift". Yes, you no longer have to worry about .so versions, but there are still plenty of reasons to test in multiple environment. "... the operating system is an implementation detail of the higher level software. It's not intended to be directly managed ..." -- yeah, so I have this proprietary SAN.. or a dual 10G cards which need to be tuned and bonded... or a mix of fast and slow disks... or even any non-trivial RAID setup... Those things are the most annoying parts of managing your own servers and unfortunately they are not going anywhere.
pslam 10y ago
There is a monumental amount of "faster horse" in modern OS deployment. Hypervisors, and their ugly cousin - secure monitors, are an example of Conway's Law. They exist because there are different groups involved in writing the kernel, user land, and server deployment. The author seems to feel that everything will concentrate at the hypervisor level - kernel and user land just a "detail" as they are single-purpose. However, you should spot that really the kernel and userland can be compressed into one layer. So why are there 3 layers at all? Then we have kernel and user land again and hey we just came full circle. Again, it's just a reflection of how the organizations are arranged. Given a single organization with authority over the entire stack, it would be a terrible waste to have that 3 layer stack.
frik 10y ago
The end of era, yes. But not what you think. Linux, *BSD and Android (Linux) are in so many devices. And new general purpose operating systems like Google Fuchsia are in the pipeline. It's the end of old closed OS, their time is over
- pjmlp 10y ago
  > It's the end of old closed OS, their time is over So where can I get PS4 BSD or those Google layers not available in the AOSP repository?

180 points by grey-area 10y ago

This makes sense. The server OS underneath ought to be a lot smaller than Linux or Windows. No interactive logon support, no display support, no printer support, no hot plugging support, few drivers, and no battery management. And, most importantly, few changes. The OS underneath ought to be simple enough to be installed for the life of the hardware. The real "operating system" is the container orchestration system.

noir_lord 10y ago
Then you put all your dependencies on top and realise how large that is and that by bundling their dependencies with those dependencies security becomes a problem keeping everything up to date so someone comes up with the idea of sharing some of those dependencies and since they are common across multiple things they need a snappy name...hmm, sharing and re-usable how about..libraries, shared libraries in fact. I jest but as an industry we do seem have an almost metronome-like tendency to lurch from one thing and back.
- walterbell 10y ago
  Lurching is a proven technique for displacement of incumbents.
- pjmlp 10y ago
  You can share libraries without having an underlying OS, a rich runtime playing OS role is enough. For example, all JEE containers allow for sharing libraries among applications, and they can be running bare metal for what I care. I can even control in which order they get loaded in each application.
  - deleted 10y ago
    [deleted]
  - MrBuddyCasino 10y ago
    Oh the horror. Thank god we don't do that anymore.
    - eveningcoffee 10y ago
      Unfortunately, or fortunately this is still what we do. Even on the web scale.
deleted 10y ago
[deleted]
Hello71 10y ago
> No interactive logon support technically doesn't come by default with Linux (the kernel) anyways > no display support can be configured out. I'm pretty sure you can even configure out the whole TTY subsystem if you want, so you can't even use a serial port to debug your madness. > no printer support who still uses parallel ports? for anything else (except usblp which basically never works) you need CUPS anyways > no hot plugging support last I checked, hotplug is mandatory on x86 (but only for certain components, so you could configure out say USB support if you wanted) > few drivers sure, you can compile your own kernel if you want > no battery management I don't really know what "battery management" means. > The OS underneath ought to be simple enough to be installed for the life of the hardware. you can already try this with Linux. then you run in to problems like "what happens when there's a vulnerability in Xen", because https://marc.info/?l=openbsd-misc&m=119318909016582 https://marc.info/?l=openbsd-misc&m=119318909016582.
- microcolonel 10y ago
  RISC-V's privilege separation model is promising for finally supporting hypervisors which probably work.
- johncolanduoni 10y ago
  > I don't really know what "battery management" means I assume he means "power management" (i.e. ACPI on x86). You can run the system without taking control of the ACPI hardware, but this is a terrible idea. It'll affect your ability to run the CPU properly, even assuming you don't care about power usage (and large server deployments sure do!). Not to mention all the nasty bugs and problems you'll run into because the hardware wasn't designed to run for long periods of time without ACPI active.
  - asadjb 10y ago
    But would that matter, given more and more of our machines are running on a hypervisor of some sort? I know it's almost mandatory for the base machine running on actual hardware, but given that most of the servers we use now are just virtualized, how bad would removing ACPI support be? Also, would it be useful to remove that?
gigatexal 10y ago
Compile a custom kernel with only the hardware in the server and then only install packages you need. Boom done.
oso2k 10y ago
This was called DOS but was deemed not sufficient for people who wanted multi-tasking.
js8 10y ago
This was already invented, it's called z/VM. You have CP, which is a hypervisor (for any operating system), and CMS, which is a single-user OS that handles talking to the user.
gaius 10y ago
This was called Mach, check it out https://en.wikipedia.org/wiki/Mach_%28kernel%29 https://en.wikipedia.org/wiki/Mach_%28kernel%29

walterbell 10y ago

There is a proliferation of OS-native hypervisors (Windows 10, macOS, FreeBSD, OpenBSD, Linux) which use hardware virtualization to isolate workloads which may be VMs, containers, unikernels, apps or even processes.

dkarapetyan 10y ago

Wise man once said the only thing new is the history you don't already know. The process is a perfectly good unit of distribution, does not require orchestration, integrates nicely with PID1, leverages OS services for getting work done, can talk over the network, start other processes, etc. The reason containers have taken a foothold is not because the process is not a good unit of abstraction and distribution. The reason is dependencies have gotten out of hand. Ask anyone that deploys fat jars to run on the JVM. They've been leveraging containers for ages now. Ask the golang folks how they like deploying single binaries. The OS is not going away. Better process isolation is the future so I'm gonna say the author has a slightly too futuristic view of things. The current container ecosystem and the churn around it is a half-measure for better process management.

xenadu02 10y ago
The history of virtualization has shown us that it is far easier to containerize existing software than rewrite it to support strict isolation. Most OSes (sans macOS) aren't really setup for sandbox-by-default either. The mainframe industry already went through this years ago. They invented virtualization and leveraged it to transition through major new CPU architectures, complete re-designs of IO stacks, and switching out entire operating systems. None of these are new arguments. Virtualization won then and it will win again. You could say the entire history of the industry is increasing levels of abstraction; containers and virtualization are just another step along that path.
pjmlp 10y ago
The day I moved from C++ back to Java and other languages with richer runtimes at work, was when I stopped caring about the underlying OS. A JEE container, is already a full OS adding the remaining services that a plain JSE installation might still lack, it doesn't need yet another layer to waste resources and add more administration work in maintenance and security.
- sqeaky 10y ago
  I write in C++ and I do not care about the underlying 98% of the time anyway. That 2% of the time I might write something that needs a kernel call, but I will make a class or some other abstraction that represents it and put all my different implementations in there. For code that cares about word size or something similarly pervasive but platform dependent there are templates and constepxr to have the compiler evaluate things. I won't be putting constants like 32 or 64 in my I will do things like "align_to(system_details::cache_line_size)" and let my compiler handle the details.

hannesm 10y ago

Processes? Seriously if there's only a single one, why would you waste resources on process management and process information? Just get over it ;) http://unikernel.org http://unikernel.org https://mirage.io https://mirage.io :D

icebraining 10y ago
There isn't a single one, there's at least one per container.

pjc50 10y ago

This seems like another point on the divergence from the traditional security model. In the 70s, the software on a computer was entirely controlled by the system administrator; the software was presumed secure and the threat was from the users. Users needed to be partitioned. In the present day, there's only one user who is also the system administrator, but the software is the threat to itself and others. True for both server-containers and mobile OSs.

digi_owl 10y ago
I dunno, seems more like the issue is that the outside attacker's path has gone from "dumb" to "smart". Where before access was done via a limited terminal protocol, these days it is done via a much richer protocol (or stack of protocols, what with tcp carrying http carrying json or some such). I don't think it was ever that local software was presumed secure. After all you wanted as little as possible to be suid, and if it was it should be as small a piece of code as possible to limit the chance of exploitable bugs.
marssaxman 10y ago
Yes! This is an argument I've been making for several years now: the user-centric security model is obsolete and unhelpful, because most computers have either one or zero users in the traditional sense. The whole unix-style reduced-privileges-plus-sudo approach that Windows and MacOS have copied is a nuisance which doesn't really solve the problem; the permissions systems in Android and iOS are a little closer to the mark. Qubes is a good step forward. The real problem is exactly what you said: we can't fully trust software, even software which is not explicitly malicious, because software can be exploited, and because software authors sometimes want to be "helpful" in ways we'd really rather they weren't. Every piece of software should run within a sandbox, and the human user should have complete control over which resources are or are not exposed to each sandbox; that's the future operating system I want to see. I did some exploration around the idea of doing this with hypervisors and unikernels (http://www.github.com/marssaxman/fleet http://www.github.com/marssaxman/fleet) but it got to look too much like rewriting all the software in the world. Containers are less elegant, but seem to be a more practical way of moving in the right direction.

digi_owl 10y ago

The title and the content are wildly divergent...

cyphar 10y ago

I don't really agree (I'm one of the developers of ocid / cri-o). Even with a container-manager-as-init you still need to administer your control plane (updates, configuration, so on). Now, you could go the CoreOS way of creating an "administration 'container'" which bindmounts / to somewhere inside the container -- that works but I would still consider it to not be any different from ssh-ing into the machine as root (it removes the need for sshd and means that you can manage the server through the same tooling you manage containers with, but the purpose is the same -- give you a general purpose OS environment you can use to administer things). Not to mention that your actual containers will still contain general purpose OS images. Also, as an aside, cri-o would be a very bad choice for PID 1. It's specifically designed so that you don't need it to be a long-running daemon (which is something that Docker cannot do and is actually a very useful feature that we hope to keep). And you can't have your PID 1 just exit whenever it wants to.

justincormack 10y ago
Even rancher no longer uses container manager as pid 1; these systems being discussed are minimal not single binary. (Only systemd is putting some "container management" features into pid 1).
- icebraining 10y ago
  Which container management features is systemd putting into pid 1?
  - cyphar 10y ago
    machinectl and systemd-nspawn. Namely, systemd now manages container processes if you ask it to. My experience with the systemd cgroup handling doesn't fill me with much faith on this topic.
    - icebraining 10y ago
      systemd does manage containers, but from what I can tell, that functionality is not in PID 1, is it?

hoodoof 10y ago

Am I the only person who thinks that Docker adds unneeded complexity?

gerbilly 10y ago
No, you are not the only one :-)
paulddraper 10y ago
The world has 7 billion people. Very unlikely none share a certain idea, particularly that one.
runeks 10y ago
How do you think Docker adds unneeded complexity? As far as I can see it simplifies a non-trivial task (automated deployment). A while ago I discovered that I can build my app (Haskell) as a Docker image, using the "stack" build tool. As far as I can see, Docker adds value by providing a layer of abstraction that makes building deployment images as easy as building the executable.

tony-allan 10y ago

There is ample evidence that we cannot write software that can be installed, run and updated in a simple and secure manner. Software is too complex with too many dependencies. Hackers are are better than us almost every time -- and once in the can bounce around at will. I just want to run my own containers or those written by others on commodity services from Amazon, Google or Microsoft. Or even spin up my own environment if it suits me. I want to be able to install and remove software quickly, completely and whenever I want. I especially want to be able to install software I don't fully trust with no fear of consequences. I want to be able to install multiple versions and even multiple instances. I want my PC (Mac/Linux/Windows) to be a thin hypervisor with everything in a container, including IO and UI. I still want open source projects on GitHub that I can fork and contribute changes to at will. I want containers to be small and ultra simple. All of this means that my "OS" only needs to manage containers and resources. All traditional elements sould be in their own container such as logging, authentication, window managers.

taeric 10y ago
You seem to be implying that containers are somehow immune to hackers... Truth is, it is relatively easy to keep a machine secure. Just not if you want it to be a convenient machine. You think you want containers all isolated from each other. Except for the uncountable times and ways you want to exchange data between them. This, at the least, includes the data of who you are, as the user.
- johncolanduoni 10y ago
  > Truth is, it is relatively easy to keep a machine secure. Just not if you want it to be a convenient machine. So the people who build high assurance systems (which currently require a ton of work) are all too focused on convenience?
  - taeric 10y ago
    Getting work done is a huge convenience. :) Specifically, remote access is a huge convenience that by itself moves the problem out of the easy territory.

winter_blue 10y ago

Library operating systems like IncludeOS are a further extension of this phenomenon: http://www.includeos.org http://www.includeos.org With IncludeOS, you don't even have a full kernel. You just have the bare necessities, and run your code in real mode in ring 0.

cheiVia0 10y ago
So your web server code has direct write access to the hard drive firmware?
- yellowapple 10y ago
  If you run it on bare metal. I reckon the point of IncludeOS is to run an application directly on a hypervisor like Xen (instead of needing a separate guest OS).

noescape 10y ago

It's happening for the operating system. Will it happen for the programming language?

erikpukinskis 10y ago

Ah, the grand cycle of layering and integration. You build a machine that does a thing, modify it to do another, and another, and another, until eventually someone says We could have a layer that did all of this! The layer is introduced, applications are rewritten to target the layer, and people slowly lose touch with what the world looks like beneath The Interface. And some things that should not have been forgotten were lost. History became legend. Legend became myth. As people target The Interface, it grows into a more and more general purpose machine. Layers of indirection build up. Once simple tasks must propagate up and down a big stack. Until one day, someone stumbles across the layer beneath. Gosh, the underlying system does 99% of what we need, out of the box. Why do we need this layer at all? We can do most of what we need with a couple single purpose tools. And the rest of the complexity can be taken up by the application layer. I don't mind doing a little more configuration there if I can get a huge performance and complexity win. And the developers, frustrated with how big and bloated their layers have been feeling, flock to this new simple tool, and they port their applications, with a little extra boilerplate and big complexity wins. And then they port another, and another. Until someone realizes We could have a layer that did all of this! And it might seem pointless, when I write it up in this snarky way, but it absolutely is not. This is the process by which we discover the fundamental building blocks of software. Each time we add a layer, and each time we take one away, we learn something new about what information is. I love it.

dfischer 10y ago
Well damn said sir. I was waiting for a sarcastic jaded snarky ending but you turned it into the essence of technology. Kudos. This is exactly it. :)
mycall 10y ago
All that complexity is entropy, information's villain.
deleted 10y ago
[deleted]
candiodari 10y ago
I don't think that's really the issue with Android/IOS operating systems. The issue is not that they can't be general purpose operating systems, it's that they aren't used that way. People seem to want applications that do one or two things at most and cost 1$ that they have to re-buy every 2 months. That does not result in people writing software that is very useful for a large part of a particular field, but takes 2-3 years to write. But the fault is in the app store and the lemon market it creates, not in the operating system.

theamk 10y ago

I feel that author is very enthusiastic, but there are some pretty serious errors: "... fight between Docker and systemd is inevitable" -- this fight has been going for a while (at least a year): https://lwn.net/Articles/676831/ https://lwn.net/Articles/676831/ "... The reason why you'll do this, rather than compose everything yourself, is compatability. Whether it's kernel versions, file system drivers, operating system variants or a hundred variations that make your OS build different from mine. Building and testing software that runs everywhere is a sisyphean task. " -- kernel versions, file system drivers, operating system variants (in form of docker daemon version) are still going to be around and would still affect containers. So you traded "test on Fedora, RHEL, Ubuntu" with "test on AWS docker, Tectonic, OpenShift". Yes, you no longer have to worry about .so versions, but there are still plenty of reasons to test in multiple environment. "... the operating system is an implementation detail of the higher level software. It's not intended to be directly managed ..." -- yeah, so I have this proprietary SAN.. or a dual 10G cards which need to be tuned and bonded... or a mix of fast and slow disks... or even any non-trivial RAID setup... Those things are the most annoying parts of managing your own servers and unfortunately they are not going anywhere.

pslam 10y ago

There is a monumental amount of "faster horse" in modern OS deployment. Hypervisors, and their ugly cousin - secure monitors, are an example of Conway's Law. They exist because there are different groups involved in writing the kernel, user land, and server deployment. The author seems to feel that everything will concentrate at the hypervisor level - kernel and user land just a "detail" as they are single-purpose. However, you should spot that really the kernel and userland can be compressed into one layer. So why are there 3 layers at all? Then we have kernel and user land again and hey we just came full circle. Again, it's just a reflection of how the organizations are arranged. Given a single organization with authority over the entire stack, it would be a terrible waste to have that 3 layer stack.

frik 10y ago

The end of era, yes. But not what you think. Linux, *BSD and Android (Linux) are in so many devices. And new general purpose operating systems like Google Fuchsia are in the pipeline. It's the end of old closed OS, their time is over

pjmlp 10y ago
> It's the end of old closed OS, their time is over So where can I get PS4 BSD or those Google layers not available in the AOSP repository?