14 ms·
My VM is lighter and safer than your container
- detaro 9y agoPaper discussed here yesterday: https://news.ycombinator.com/item?id=15600596 https://news.ycombinator.com/item?id=15600596
- fredrb 9y agoMy bad
- exabrial 9y agoThis is pretty cool tech, a "minimized" OS and support libs
- graton 9y agoIntel has something like this: https://clearlinux.org/features/intel%C2%AE-clear-containers https://clearlinux.org/features/intel%C2%AE-clear-containers They call them containers, but it is actually a VM.
- bryanlarsen 9y agoThe two initiatives are quite different, IMO. AFAICT ClearContainers only supports running your current kernel and virtualized hardware in their VM. That assumption gives them some massive speedups. LightVM supports running arbitrary kernels, and can be really fast when running a very light weight micro kernel.
- kraemate 9y agoHow many custom kernels are out there? Given the already sad state of most user-space software, do we really want a proliferation of supposedly light-weight kernels?
- convolvatron 9y agoI agree with your point about 'supposedly' light weight. and its lot clear how much weight really costs the kernel's user. but I'm pretty reluctant to agree that 'last years linux with a random bunch of patches' is the ultimate solution, or even the best solution available today. the one really interesting thing that VMs give you is an abstract diver model. it might not be the absolute best or higher performance, but if a kernel or unikernel is designed only to be run in an VM environment, on a server, then the giant mass of buggy drivers just goes away. It may even be possible in some circumstances to run with just a VM network device and nothing else. I personally think there is some fruitful work to be done around the kernel interface to admit better scheduling of i/o in highly thread-parallel environment. But I have to imagine there are some radically improved security architectures for server-in-a-VM that doesn't involve a Linux userland, firewall, package manager, etc etc
- tabeth 9y agoLooks very interesting. One thing I've been wanting is a VM that would contain a "minimized" version of an operating system, plus a single program, in addition to the programs dependencies. I've looked into using Docker to do something like the above, but it didn't seem straightforward a few years ago. LightVM seems like it could be an option in the future.
- inetknght 9y agosee also #include <os>
- yeukhon 9y agoIsn’t that what Busybox supposed to be? There is Damn Small Linux. DSL was very pleasnt to play with! So sad it is no longer maintained.
- kbenson 9y agoSmartOS's LX branded zones seem kind of cool to me, and may hit an interest point for you. Basically, SmartOS is an OpenSolaris variant where they use Solaris Zones for docker images, but they've implemented a Linux syscall table API so you can create Linux "compatible" containers from a Solaris kernel (and also full access to ZFS, etc). You can see a talk here[1] where Bryan Cantrill goes over the reasons you might want to use containers, SmartOS and how LX branded zones. Warning though, he's an extremely engaging speaker, so not only are you likely to watch all of that 40 minute talk, but you might feel like binging on some other talks of his, so here you go.[2] 1: https://www.youtube.com/watch?v=coFIEH3vXPw https://www.youtube.com/watch?v=coFIEH3vXPw 2: https://www.youtube.com/watch?v=-zRN7XLCRhc&list=RDQMdV_4MNC5Crw https://www.youtube.com/watch?v=-zRN7XLCRhc&list=RDQMdV_4MNC...
- axelfontaine 9y agoThis is exactly what we provide at Boxfuse (https://boxfuse.com https://boxfuse.com), with the ability to run it on VirtualBox, Hyper-V and AWS.
- odammit 9y ago
- mrmrcoleman 9y agoThis is almost exactly how Hyper.sh works under the hood: http://hyper.sh http://hyper.sh Disclaimer: I used to work with Hyper.sh.
- _Marak_ 9y agoWhy are you no longer working with Hyper.sh?
- mrmrcoleman 9y agoI was hired as an external for a specific project which was completed.
- frou_dh 9y agoMy personal long bet has been to completely ignore Docker etc. Seems that has bounced between plain ugly apathy and foresight.
- RubenSandwich 9y agoI personally think Unikernals will overshadow Containers in a few years. They just in my mind offer a model that is much more security and speedy. I give them a few years because Unikernals still need to figure out better tooling; debugging, producing and deploying them is currently difficult.
- tjbiddle 9y agoA few years is a lifetime in tech; why not learn what is new and see how if it's applicable to you and improves things now? You'll come out with more experience, can throw it on your resume, and very likely be more prepared for the next tech to come along.
- unilynx 9y agoIt seems pretty unlikely that unikernels will ever replace containers. Containers allow you to run almost all of the past decades of software with no or little modification (and most modifications are done in bash), a unikernel is more like a new OS that applications need to be ported to, without the full POSIX API (eg, no subprocesses) - it's probably easier to port a multiprocess server (eg most SMTP servers) from Linux to the NT API than to port them to a unikernel
- pjmlp 9y agoNot necessarily, on programming languages with nice standard libraries/runtimes it is just a matter of running them bare metal. A JEE application doesn't care if WebSphere is running on OS XYZ, bare metal or an hypervisor. In a way I consider POSIX C's runtime that should have been made part of ANSI C.
- pjmlp 9y agoI think it would be quite cool, but it needs to become the next fashion in order to go mainstream. After all we on Java world were already doing that with application containers, it would be just a matter of running them bare metal.
- andrewstuart2 9y agoContainers are not about safety or lightness IMO. Rather, the reasons I prefer them are better insight and usability. Since the same kernel is running multiple applications, it has insight into how much RAM has been requested, and how much CPU is being used, and can thus be more efficient in packing processes onto nodes. If you try to build this with unikernels, you will end up turning your hypervisor into a de facto kernel. As it is, hypervisors simply preallocate resources and in pretty much every case must overprovision for peak usage, which requires more unused capacity. As for usability, I mean that you get many of the benefits of VMs (namespaces and resource constraints), but still with the added benefit of the kernel introspection.
- craigyk 9y agoI think what you describe most people lump into the "lightweight" characterization.
- andrewstuart2 9y agoUnder that definition, then, VMs would not be lighter than containers, because they require overprovisioning and don't (by deliberate design) dynamically communicate memory or CPU usage to the hypervisor.
- inetknght 9y agoLinux supports hot-adding memory. Since it can do that, as a guest, so can any other VM. Overprovisioning might be required, but it's only required because of software limitations, not by hardware. I would venture a guess that could be solved.
- andrewstuart2 9y agoHot-adding memory is essentially have the equivalent of the mmap kernel syscall, but there's no ability for the VMs to return memory that they no longer need, as you have with the munmap syscall. So you may start out without overprovisioning, but eventually you will hit peak hours, redistribute your VMs, and be overprovisioned again unless you reboot machines after peak hours. Even if you did add a capability to compact and dynamically stop using hardware memory into linux, then signal the hardware layer that it's not used, you're again just implementing another feature provided by standard kernels for decades. Not to mention this feature seems basically coupled to virtualized hardware.
- otterley 9y agoThe perplexing thing to me about this paper is that "containers" are launched merely through a particular type of clone(2) semantics (and clone(2) implements fork(2) under the hood on Linux). So the process of launching a new container should take around the same time as it takes to perform a fork. The fact that it doesn't in their analysis is surprising. So the question for me is, what is the paper measuring, exactly, when they talk about Docker's relative performance? Are they including Docker API overhead in their instrumentation? If so, that sounds like an apples-to-oranges comparison.
- fpoling 9y agoTo start a container Docker does a lot of expensive setup like preparing network stack, volumes, logging etc. One can trivially see how expensive is that just by timing the command line and dropping expensive Docker options. On my laptop: $ time docker run --rm ubuntu true real 0m0.641s user 0m0.036s sys 0m0.018s $ time docker run --rm --net none --log-driver none --read-only ubuntu true real 0m0.399s user 0m0.039s sys 0m0.018s Compare that with the unshare utility that one can use as a Docker replacement for simple containers (options create all new namespaces that the utility supports including user namespaces): $ time unshare -m -u -i -n -p -U -C -f -r /bin/true real 0m0.029s user 0m0.001s sys 0m0.009s
- jsjohnst 9y agoHave you ever straced a Docker run or investigated what it does on container startup? I recommend doing so if not!
- apeace 9y agoThis is interesting, but means very little to me as a heavy user of Docker. I don't care about boot time. I care about 1) build time, and 2) ship time. In Docker, they are both fast for the same reason. At the top of your Dockerfile you install all the heavy dependencies which take forever to download and build. As you get further down the file, you run the steps which tend to change more often. Typically the last step is installing the code itself, which changes with every release. Because Docker is smart enough to cache each step as a layer, you don't pay the cost for rebuilding those dependencies each time. And yet, you get a bit-for-bit exact copy of your filesystem each time, as if you had installed everything from scratch. Then you can ship that image out to staging and production after you're done locally--and it only has to ship the layers that changed! So this article somewhat misses the point. Docker (and its ilk) is still the best tool for fast-moving engineering teams--even if the boot time were much worse than what they measured.
- Pyxl101 9y agoThe article is about using and optimizing VMs and containers at runtime, not at development time. Whether Docker provides development benefits compared to other approaches is orthogonal to the runtime performance concerns being discussed in the article. People use containers and VMs for multiple purposes. Some people might only care about development speed; other people might care a lot about runtime performance and will find this article relevant. The article's point is that, if you need a lightweight runtime isolated environment, VMs are viable, competitive with containers, and have unique isolation benefits.
- yebyen 9y agoDefinitely. There are different use cases even within the same shop. One of the major problems with getting containers adopted where I'm at is that the ops folks want to have a production story for containers before we start learning how to use them in development. Many of the concerns are orthogonal! We are just using them on the app-dev side of the fence to manage our configuration drift so that regardless of what app you're trying to start, you can do it alongside of whatever other apps you're already running. We use minikube with the ingress addon to help us fit some moderately complex constellations of servers onto our laptops, without having to know Ansible or learn to configure Nginx (or need to twiddle the Nginx config every time we're changing to work inside of a different context, or know a lot about Kube, or requisition additional EC2 nodes, or really anything after setting up Deis other than "git push deis".) Before they will call this development configuration supported and allow us to take advantage of cloud resources (S3 buckets to address our scaling and other concerns, basically so we can run Deis in minikube without starting from scratch again every day...) the enterprise architects want to see a plan to run "a multi-AZ scalable environment to run containers." It was at that point in the conversation that I realized, we were having two totally separate conversations in a single thread. There is so much overhead to get things into production in our organization that we don't even want to broach the topic of putting containers into production use, but in the minds of InfoSec and the architects, they see it as inevitable that if we're using containers in development, we're also going to use them in production. I'm on the fence.
- arca_vorago 9y agoAs a senior sysadmin who has had to deal with all the crap devs throw at me, I skipped the container craze to focus on VM security and other issues. Give the devs a nice powerful boxen to use (APIs in many hypervisors these days)so they can do the provisioning themselves. Without all the added complexity another tooling like docker requires. Real world benchmarks I ran indicated perf differences were negligible, so devs seem to mostly like docker cause they don't have to do sysadmin stuff... but I can do all the same things in a VM, plus much more. A big shoutout to the people at Proxmox while I'm at it.
- rconti 9y agoAlso a sysadmin, I've inevitably found that the environment needs/expects a number of services, and it just feels like docker is ill-suited to these use cases. Frequently even the devs themselves want multiple services and can't tolerate stateless systems anyway, but at the very least we have to deal with logging and the like. I'm not saying it can't be done, but I've never found a developer who had a particular desire to implement containers, and even if they did, their concerns don't represent the entirety of what's involved in running infrastructure.
- arca_vorago 9y agoI feel like the increase in devs in general and in business have made them more likely to follow bandwagons and fads more easily than they used to, which just makes our job harder.
- tootie 9y agoYou mean like some private cloud software? I'm in this same boat from the dev side trying to negotiate a sustainable option that doesn't slow us down.
- arca_vorago 9y agoForget the word cloud. The cloud exists on servers. What you are talking about is servers. I mean build a nice beefy 2 cpu supermicro rackmountable 2-4u server with a shitton of RAM and fast disks and a duplicate for HA which is then backed up properly. It is a dev and a sysadmins playground paradise. I do prefer to keep prod and dev on separate physical machines though for an added layer of separation. One more thing. People seem to think colocation isn't worth it anymore... well let me tell you, they are very often wrong! Colo is a great solution depending on the problem. You get the closeness to the core backbone nodes, the security of a DC, and these days often at very competitive prices. You can get security cages, full racks or you can half racks and sometimes the DC has smaller etc to keep the cost down as well. If thats not an option, you can do dedicated servers in a DC. The difference being in a colo the hardware is yours, in a dedicated setup the hardware is part of an offering by the provider. I have done this at many places as a stopgap until the colo server arrives (rackspace for example). I would really suggest for real dev where you are compiling often and doing many things that stress systems, stay away from VPS's. It's not nice to other users and you don't get the perf you need/want anyway. This kind of stuff is what makes me excited about 32core amd cpu's on the way. This time I wont have to do a quad cpu system to get 64 cores, just 2! https://www.supermicro.com/Aplus/motherboard/EPYC7000/H11DSi.cfm https://www.supermicro.com/Aplus/motherboard/EPYC7000/H11DSi...
- simonjgreen 9y agoIf you're stuck in the middle, check out VIC (vsphere integrated containers) for a best-of-both. http://www.youtube.com/playlist?list=PL7bmigfV0EqTsggWvOpPWYdlwVwTj2_K7 http://www.youtube.com/playlist?list=PL7bmigfV0EqTsggWvOpPWY...
- srigi 9y agoIf unikernel image boots in 4ms, it can be used to directly hande incomming traffic in some cases. Does anybody have an experience with this?
- yaantc 9y agoThere's a prototype Mirage based DNS server that does that, Jitsu. See there: http://www.skjegstad.com/blog/2015/08/17/jitsu-v02/ http://www.skjegstad.com/blog/2015/08/17/jitsu-v02/ Quote: "Jitsu - or Just-in-Time Summoning of Unikernels - is a prototype DNS server that can boot virtual machines on demand. When Jitsu receives a DNS query, a virtual machine is booted automatically before the query response is sent back to the client. If the virtual machine is a unikernel, it can boot in milliseconds and be available as soon as the client receives the response. To the client it will look like it was on the whole time."
- anitil 9y agoHoly Moly, sometimes I forget just how fast computers can be.
- thegabez 9y agoKeep your VM's, I'll stick with Docker/K8's
- xellisx 9y agoDoes it have any ability to use USB hardware? Something I looked into in the past was trying to use LXCs to do a multi USB WiFi adapter router to connect to multiple networks, which might be using the same 1918 netblocks. Unfortunately, LXCs had issues of "bridging" USB hardware into the LXC.