21 ms·
I've had this sneaking but hard to articulate suspicion that datacenters, bare metal servers, VMs, operating systems, containers, OS processes, language VMs, an
by dopylitty 7y ago
I've had this sneaking but hard to articulate suspicion that datacenters, bare metal servers, VMs, operating systems, containers, OS processes, language VMs, and threads are all really attempts to abstract the same thing.
You want to run business code in a way that's protected from other business code but also able to interact with other business code and data in a well defined way.
I also have this sneaking suspicion that new generations are re-inventing the wheel in a lot of ways. If you have minimal containers running on a hypervisor how is that different from processes running on an operating system? You have all these CPU provided virtualization instructions to protect guests from each other and the host from the guests but there's no reason those instructions couldn't have been developed to protect processes from each other. You have indirections to protect one guest from accessing another's storage but there's no reason processes couldn't have the same protections (and they do in many operating systems). Why container orchestration and overlay networks instead of OS scheduling and IPC?
I'm sure people in academic computer science have already published many a paper about this but it feels different seeing it from inside an IT organization where people don't seem to apply the lessons from older technologies to newer ones and we end up in this constant churn of reinvention which, as far as I can tell, is mostly a way for people, both in management and in the trenches, to keep their jobs, at least until you're "too old to learn new things" and pushed aside.
- BubRoss 7y agoThreads are not the same because they are separate computation in the same address space. At some point though, permissions to disk, network, screen, input devices, etc. need to be an integrated part of computing for anything that runs more than one program. Take a look at QNX though, at some point it will be reinvented and heralded as a breakthrough, 40 years after it happened the first time.
- dopylitty 7y agoThat's a good point. But as I think about it in a sense threads are also a form of isolation in that they allow you to isolate units of business code so that they can run independently and be (potentially) protected from other business code monopolizing a shared resource while not actually doing work.
- BubRoss 7y agoThreads are not a form of isolation, they have nothing to do with ' business code ' and they offer no protection at all. I'm not sure what construct you are talking about, but it is not threads.
- GauntletWizard 7y agoProcesses running on the same host are a "separate computation in the same address space" - They share a filesystem namespace. Pods running on the same kubernetes cluster are a "separate computation in the same address space" - They share a DNS namespace. Servers running on the same internet are a "separate computation in the same address space" - They share an IP namespace. These are the same abstraction at multiple levels, and that's okay, because you often do need that. The L1 and L2 cache are crystaline to inmemory and redis caches in distributed systems. Components with the same shape show up at multiple levels. This is okay. What's important is that those levels each serve a purpose, and that you reduce the abstraction to be as simple and meaningful as it can be.
- jacques_chester 7y agoA good analogy is network layers. The problems and solutions handled at the network layer resemble those at the internetwork layer, which resemble those at the transport layer, in turn similar to those at the application layer. Convergent evolution doesn't mean we should all become the same organism.
- BubRoss 7y agoProcesses exist in separate virtual memory spaces. Threads in the same process exist in the same virtual memory space. This is a fundamental principle of virtual memory. Mixing this up with a filesystem or any shared resource is completely nonsensical. Naive oversimplifications like this indicate a lack of understanding of how and why computers are architected as they are in the first place. Virtualization could be said to be an extrapolation of virtual memory and in fact does offer sub virtual memory, but to equate threads, processes, virtualization and shared resources as the same thing is grasping for simplicity that isn't there by making comparisons that aren't true.
- sansnomme 7y agoBecause current model of processes frankly sucks. If I give you random binary would you run it? You can talk about sandboxing, lecture about permissions and principles of least privilege. But that still doesn't answer the question: can it run hostile code without having side effects on the rest of the system? Other than the newer web tech initiatives like WebAssembly/JS sandboxing, there do not exist any other technology that allows easy compartmentalization with somewhat reasonable security isolation out of the box for existing server code. Containers are not perfect, and hypervisors/unikernels are a pain to deploy on commodity cloud services but we are slowly getting there. (Also if you notice, newer generation of operating systems like Android, iOS, Fuchsia are moving towards strong sandboxing with every single thing locked down where possible. There is an impedance mismatch between the Unix model and current demands. You can't migrate overnight so you build technologies to work around it. From far away, Windows and Fuchsia are very similar, but Windows have to contend with decades of code compatibility needs even if they want to start shoving everything into a locked down app)
- taeric 7y agoThis gets silly, though. If you are going for full isolation where applications can't just let the user have data, then you have to coordinate every app talking to every other app. This is why the share button on a phone is ridiculous. Instead of just copy to clip board, you get a ton of options. Is it safer? Hard to say. I can do less and have less capabilities without more active work from the developers. There is something to be said for this, of course. But it seems a losing game. I contend there is no technical panacea to security versus convenience.
- GauntletWizard 7y agoThere is no technical panacea, but containers are a useful abstraction. Containers bring back the original intent of the unix process/user security model, allowing you to scope each process to have it's own 'private' filesystem to get around the broken way executables are built, packaged, and linked. This doesn't save you if you do dumb things, like have open ports that allow administrative access that are exposed to the rest of your (potentially malicious) system. They do force you to think about some of the questions, though, like "Where is my data stored and how do other tools interact with it?" Most of the problems that people complain containers don't solve are ones that are open ended, and have no definite solution.
- jeieo9nkfkr 7y agoI have a sneaking suspicion this has a lot to do with business finding a “market niche” more so than reasonable engineering. They wrap it in semantics they know from marketing physical products, and hey the general public is ignorant...poorly informed ...I mean cool with it... give the people what they say they want when we decide what’s available in the first place... I mean what fits our business niche. If that feels a bit circular, human society is a bit circular. I’ve become convinced we’re doomed as so many cannot see the difference between objective physical reality and metaphor/analogy. Even a lot of educated people struggle with the emotional dissonance despite getting through the homework. Aristotle and scores of other philosophers wrote of it too. The concern being if the general public knew there really weren’t Gods on the mountain, they’d be pretty fucking pissed. So we’ll conveniently “let them have” their myths and legends.
- regularfry 7y agoFrom memory, I seem to recall that Genode takes this to its logical conclusion: every process is isolated with virtualization primitives. As to why: isolating processes the old way needs jails to work properly, BSD lost the popularity contest, and Linux jails didn't get secure enough before VMs and containers took off.
- sayhello 7y agoDid the BSD jails provide enough isolation? Would memory from a process truly be isolated from another? How about FD’s? Could there be side channel attacks? At what level? Perhaps the whole kernel design needs to be revisited, which I assume is what’s being taken on by Fuchsia.
- braindeath 7y agoI’ve seen no evidence that Fuchsia is bringing much more than 90s era embedded OS design. Just because it’s not Linux doesn’t mean it’s new. Not that this is a bad thing, but someone please correct me if there is something fundamentally recent in Zircon.
- marios 7y agoThe BSD jails / Solaris zones approach is not the same at the path taken by Linux. Linux gives you a facility to isolate network, a facility to isolate process views etc. You put them all together and you get a "container". The former starts off by having a container primitive that can't do much because, well, it's contained from everything. You then proceed to give it access to the network, the filesystem.
- sayhello 7y agoGood point. The current widely used kernels seem to make some assumptions about ‘owning’ the hardware and implicitly trust any running programs at a certain level. There is a need, however, for some pragmatism. While a capabilities-based OS could be the right solution, it doesn’t exist today. What we have is code built for some flavor of POSIX. People demand cost efficiency today. So, engineers do what’s practical: build a multi-tenant system running code that expects to be the owner of the hardware. It works good enough! For most people, the cost efficiency math works out. The sand boxing overhead is worth it, as it pales in comparison to the headcount required to manage your machines, or even VMs, if you replace them with Serverless products.
- jacques_chester 7y agoWhat you're seeing is the endless see-sawing between "performance needs to be higher" and "isolation needs to be stricter". Each alternative lands on a different balance at different times. Partly because of their own innate qualities, partly because of path dependency, partly because a "new" approach can sometimes reveal previously-unnoticed assumptions and partly because the ratios of CPU, RAM, disk and network performance are ever-shifting and thus favouring this or that architecture in the moment. What tends to happen is that performance gains draw the first wave to a "new" technology. Then as the bulk of the industry begins to move, isolation, robustness and backwards compatibility become relatively more important. These impose overheads that will eventually be swept away by the next "new" technology. There are still qualitative differences, though. Where you draw the firm lines in architecture affects the range of possibilities. I can place tenancy boundaries "anywhere", but in practice a VM hypervisor will be able to provide a firmer boundary than asking people not to share passwords.
- no_wizard 7y agoWhat I’ve always failed to Understand is how FreeBSD jails[0] never got very popular (discounting the fact that FreeBSD isn’t very popular on the whole from what I can tell) but Docker is huge. I personally think jails are superior in implementation in that it requires no other abstractions on top of the OS. The only thing I can surmise is that Docker might have a better secure default, but improvements to Jails could emulate that, and following best practices as well should. Beyond that I’m still baffled how this didn’t take to the mainstream. [0] https://www.freebsd.org/doc/handbook/jails.html https://www.freebsd.org/doc/handbook/jails.html
- beaconstudios 7y agobetter marketing I would assume.
- dasil003 7y agoBecause Docker has better dev marketing. Some sizable portion of developers actually believe Docker invented containers.
- 0xcde4c3db 7y agoThis, pretty much. Nearly everything written about Docker basically treats it as synonymous with containers. This is also incredibly annoying if you're already familiar with the general concepts of containers and actually want to read something about Docker in particular.
- zzzcpan 7y agoJails are more like runC, container tech behind Docker.
- jayd16 7y agoBut they are popular... Jails and chroots are used heavily in iOS and Android. I would say its one of the fundamental differences between mobile apps and desktop apps. Its not enough though. You need docker to handle the DNS, routing, port forwarding etc. if you want to emulate a pocket cloud on a dev machine. I suppose the networking aspect could be done for jails as well but docker and docker-compose just seem more polished in this regard.
- sedachv 7y ago> I've had this sneaking but hard to articulate suspicion that datacenters, bare metal servers, VMs, operating systems, containers, OS processes, language VMs, and threads are all really attempts to abstract the same thing. There is a very easy way to articulate it: they are all ways of virtualizing different facilities. Unix processes virtualize the user-mode processor registers and the address space. POSIX threads only virtualize the processor registers and the stack. Linux namespaces let you pick and choose which facilities you want to virtualize (storage, PID namespace, IPC namespace, etc.). The abstraction is related to what is being virtualized: everything that is virtualized is a namespace (this is what naming things refers to in Phil Karlton's quote "There are only two hard things in Computer Science: cache invalidation and naming things," not to the literal naming of variables in computer programs). Another way to go instead of virtualization is capabilities.
- user_error 7y ago> this is what naming things refers to in Phil Karlton's quote "There are only two hard things in Computer Science: cache invalidation and naming things," not to the literal naming of variables in computer programs That is mighty interesting. Do you happen to have a source for this? It's the first time I've heard it being put in this particular way.
- Yetanfou 7y agohttps://skeptics.stackexchange.com/questions/19836/has-phil-karlton-ever-said-there-are-only-two-hard-things-in-computer-science https://skeptics.stackexchange.com/questions/19836/has-phil-... Several references to the quote "There are only two hard things in Computer Science: cache invalidation and naming things" can be found, such as Martin Fowler's blog and others. I'm unable to find the source of this quote, has he ever said it? - As his only son, and colleague with him at Netscape from 95-97, I can attest that my dad did indeed throw that quote around, on more than one occasion. I'm fairly confident that he originated it (he was fond of coming up with clever quippets), though I haven't been able to figure out how it disseminated so widely over the past couple of decades. I'll keep looking around in old web archives and mails to see if I can dig something up.
- dustingetz 7y ago“function” for suitably principled definition of function
- reaperducer 7y agoI also have this sneaking suspicion that new generations are re-inventing the wheel in a lot of ways. Suspicion fades away, and the older you get, the more often you see this happen.
- circute 7y agoDid I write this comment? I've had the same feeling. I work on securing VMs in a hypervisor and part of me wants to look at ring 0 to 3 instead of ring -1 to 0 and see if I can fix that instead of bloating cpu specs with more rings.
- louwrentius 7y agoI think you could build a secure application environment with just bare metal. The only hip tool I would use is Ansible after paying my respects to cfengine. No virtualisation No containers No overlay networks A few VLANs perhaps. If I may. It can be done. Just a few dedicated people. The kind of people colonel Kurtz talked about...
- drybjed 7y agoI'm working on something like this with the DebOps[1] project. It's a set of Ansible playbooks and roles that lets you manage Debian environments. I aim for it to be agnostic - roles can be used effectively both in LXC containers, virtual machines or bare metal. You can start with a few machines with Debian netinst + SSH installed, and build them up with the services you want. Applications are installed either from Debian or upstream repositories (via APT), downloaded directly from upstream with proper signature verification, or compiled from git sources. There are no OS images, therefore no containers or VMs are strictly necessary. It seems that in the age of Kuberenetes projects like this are a niche, but if somebody is working with on-premises infrastructure, bare metal hosts, a good set of tested Ansible roles which you can use as a base for your own deployment might come in handy. I haven't found anything close to what I had in mind back then, so I started my own, going at it for 6 years now. Hopefully it will be useful to somebody else at some point. :-) [1]: https://github.com/debops/debops/ https://github.com/debops/debops/
- louwrentius 7y agoThank you for sharing this!
- barbecue_sauce 7y agoIs Ansible still hip or has it gone the way of MongoDB?
- demosito666 7y agoAnsible is still a practical tool for some of configuration management tasks, as opposed to mongo which I have hard time finding a niche for.