5 ms·
Are Containers the Right Answer to the Wrong Question?
- sturadnidge 12y agoWherein the author argues that requiring a single OS kernel for multiple containers is somehow more restrictive than MirageOS, which requires every application to be written in OCaml? The overall concept is not without merit, the OSv site (http://osv.io/resources/ http://osv.io/resources/) has some more interesting material. I'll still be sticking with Docker though.
- mackwic 12y agoWhat's the big deal with OCaml ? I found it to be a very nice, powerful, and enjoyable language.
- dmpk2k 12y agoThere's nothing wrong with OCaml specifically. The problem is the overall argument, which seems to fundamentally misunderstand why machine virtualisation is used.
- vidarh 12y ago"The big deal with OCaml" is that most people have tons of code that isn't written in OCaml. Projects like MirageOS and ErlangOnXen are fantastic and I hope we see more of them, but getting to the point where these kinds of runtimes are viable for most people is a long road.
- amirmc 12y agoThey are viable for people starting afresh. Not everything that comes along has to care about legacy systems so the advantage goes to those creating new tools/products/services with the advanced approaches. In fact, I'd argue that such step changes wouldn't be possible if we always had to be compatible. This is pretty much 'circle of life' stuff.
- amirmc 12y agoThe point about Mirage (and it's ilk) is that you can retain the benefits of the Hypervisor (security etc), while also gaining the benefits of streamlined deployment. Those who can or choose to use such approaches will benefit from the best of both worlds. In general, I see things like Docker/Mirage/Xen to be complementary, rather than antagonistic. A good example of this is Google's Docker support, which is via a custom VM. https://developers.google.com/compute/docs/containers https://developers.google.com/compute/docs/containers
- ithkuil 12y agoVirtualization is very good at handling consolidation of legacy systems. Which means that your 10 old servers, running various unix and windows versions and hardware whose spare parts are difficult to find, can be easily replaced by a single modern machine, usually not even particularly beefy by modern standards. It solves a lot of problems in small to medium companies; it's an answer to a desperate call for help of sysadmins too tired to hunt 15 years old cpu fans on ebay, and managers hearing always the same infernal stories about what it's like to do anything down in the basement. It solves the problem of the same small to medium companies having to buy new hardware for each application they buy, usually because the vendor tells them to do so. Many software started to get shipped as a whole OS stack, because that was easier for the vendor to support. It solves many other problems, but let's stop here. These problems are going to stay, and so will virtualization. However containers offer a way to isolate your environment so that they can survive longer and coexist better with other apps. If you build your environments using those building blocks, then after 10 years, you'll have a slightly smaller mess (which you can still handle with virtualization). If you really have a lot of machines then, you need to have a good system for managing your hardware, for bulk installation etc. You have to manage new kernel and container management software rollouts in more or less the same way you'd need to manage hypervisor rollouts. But since you have a lot machines, which means that you encounter hardware failure more often, the relatively higher frequency of kernel and container management software updates has still a smaller cost than virtualization overhead (provided you have enough machines). Or put in another way, virtualization overhead is negligible if you're a small fish. It's cheaper for you to buy more machines or a stronger machines if all you need is to cover for that overhead.
- KaiserPro 12y agoAs someone who battles legacy at scale everyday, Its rarely the platform that is the trouble. For example, I have a nasty little calendar script that I have to move. The rest of the app is long dead, but this one little worm keeps on biting. It uses regex to parse XML. (insert cuthulu joke here) its based on php5.old I need to move it to 5.new+secure. The OS has an update function that is know, tested, documented and works. I can install a new RPM/deb and be fairly sure it'll work if it installs Standalone apps, in the way that people are building them wont. Conatainers rely on the underlying operating system for various things. Any exceptions are bundled with the app. a true VM is free to be moved to a new host, a container is not. You are hamstrung to that machine and possibly that config. So If you built the farm on docker, you're stuck with it, on that hardware. unless you want down time, or was to re-engineer things
- pointernil 12y agoI think "Erlang on Xen" is as well a nice example pointing in this direction. In this case it is not a application though. It is a runtime environment... something in between a whole VM and a container? http://erlangonxen.org/ http://erlangonxen.org/
- nwalfield 12y agoThis post is disingenuous. In particular: [Cloud Operating Systems] enable the vision of light and fast application servers, while preserving the superior security and flexibility of true hypervisors. Hypervisors are only secure if you don't need any sharing. If you have two applications running as separate guests and they need to exchange data, then all of the security issues that you'd have on a normal OS return, but they are harder to solve, because the Hypervisor doesn't provide much support! As an extreme case, consider running all of your applications on your Desktop in their own VM. Now, you are in your web browser and you download a Word document that you want to edit. Somehow, you have to transfer this document to the Word Processor VM to edit it (they have different file systems so you need to use something like scp). If, after you're done editing it, you need to send it to someone via email, then you'll have to copy it to the Email VM. It's not how to see how this seriously impacts usability! When sharing is required, then a Hypervisor is not helpful. That's not to say that Hypervisors are useless, as the author claims his opponents say. Hypervisors are good for running different operating systems on the same hardware and they are good at isolating the different OSes from each other. However, once any sharing is required, hypervisors are not the right tools for the job.
- amirmc 12y agoI'm confused. What you're describing seems equivalent to just connecting a machine to the internet/network (ie you have security concerns). How many scenarios are there where you have two guests on the same host that must communicate? Shouldn't those applications/VMs be architected in a way that you could simply move one of them to a different host without breaking? I guess I'm confused about what you mean by 'sharing'. Edit: As for your extreme example, I believe Bromium (ex-XenSource founders) is doing what you describe. http://www.bromium.com/products.html http://www.bromium.com/products.html
- throwaway6687 12y agoThere's also Qubes OS, which unlike bromium can actually be downloaded and tried out by regular people. http://www.qubes-os.org/ http://www.qubes-os.org/ The bromium site sounds nice, but... there's no download button, no "buy" button, or anything to suggest that it's a ready product. It's also very short on actual technical details of their solution.
- phkamp 12y agoBefore Linux adopted "Containers", jails were invented in FreeBSD (by yours truly) and they ran Linux Apps just fine, thankyou. And SVR4 and SCO binaries too for that matter. The idea that your kernel can only give you one syscall-API is so 1990-ish.
- KaiserPro 12y agoWell the '90s are very much back in fashion. perhaps jails need's a little boyish mustache? also; hurrah for jails.
- threeseed 12y agoIt's all about end user experience. Docker wrapped up containers in a shiny, red bow and handed it to developers who aren't that interested in digging around the OS to set it up themselves.
- lelf 12y agoThe only thing that is more shiny about docker is the site with bootstrap. Jails are as easy (if not more) to setup and use as containers. The only my wish for FreeBSD is some easy way to do pkg install $PKG --and-run-it-in-jail=jail22 --thxbye
- justincormack 12y agoLinux was simpler to emulate back then, it has grown in complexity and the emulation has not kept up, although it does work for some use cases. Having worked on testing for the NetBSD Linux emulation (its getting better, didnt help having no tests) there is quite a lot missing (epoll for example is not yet emulated). FreeBSD still appears to only have 32 bit emulation which rules out most modern Linux apps.
- thejdude 12y agoWhat is it that makes "most modern Linux apps" not run on 32bit? Last I checked, pretty much all packages were available for both x86 and x64. If you're talking enterprise use, sure, some app servers might need more than a few GB, but that's a memory problem, not strictly a bitness problem.
- KaiserPro 12y agoAnd so the circle is complete: TL;DR: this is timeshared mainframes all over again. I'll dust off my pipe and white technician's coat. In the beginning there were batch run mainframes, and it was good. And Lo!, did the bean counters notice thy ever under utilized CPU cycles. So became time sharing mainframes. The masses rejected the cold noisy cave of the server room and so it was decreed that the terminal shall forever more be remote. Thy ever indiscriminate shrink ray doth break the bonds of the mainframe, and set forth the world into mini computers. "Hark!" said the masses; we demand the Personal computer. The masses were mollified, until the becoming of cheap internets. "But wait!?" they cried we require some sort of network computer. Time passed, and a tower of babel was conquered by XML, JSON and eventually the cloud.
- mp8 12y agoHate to be "that guy", but that should read "And Lo! ..." http://english.stackexchange.com/questions/38741/use-of-and-lo-in-a-sentence http://english.stackexchange.com/questions/38741/use-of-and-...
- KaiserPro 12y agomuch obliged.
- jasode 12y ago>TL;DR: this is timeshared mainframes all over again. The article was reiterating the advantages of virtual machines -- which handles different operating systems. I thought the timeshare concept on mainframes was user-&-application-level time slicing and not os-level. (E.g. an IBM 370 does not simultaneously run OS/360 and MVS operating systems in isolation.) Therefore, the closer analogy to mainframe timesharing would be Citrix Xen desktop or Microsoft Remote Desktop Session Host.
- KaiserPro 12y agoTHe hypervisor concept is something that IBM invented and shipped with the 360-40. You can run lots of code from the 70's with little to no modification on a z series mainframe, because its visualized. However My analogy is based on the idea the cloud is effectively a mainframe. They are espousing the joys of single application VMs that effectively rely on the host to do the time-sharing. Something very reminiscent of early timesharing OS
- rwmj 12y agoAlso, containers don't do live migration. If you can transparently migrate VMs around your datacenter then you can consolidate VMs onto a few hosts to save electricity at night, spread them out during the day to maximize performance, or move them for scheduled hardware upgrades.
- amirmc 12y agoI know that live migration is possible but I'm curious how many people actually do this. I hear much more about spawning/culling VMs quickly, than I do about live migration.
- spydum 12y agoPretty much all day on busy clusters. VMware esx with DRS does this to rebalance resources.
- regularfry 12y agohttp://bigv.io http://bigv.io does, it's used every day. Qemu live migration plus storage migration means (in theory) zero-customer-downtime hardware maintenance and load balancing.
- skj 12y agoGoogle Compute Engine has live migration between datacenters. http://www.rightscale.com/blog/cloud-industry-insights/google-compute-engine-live-migration-passes-test http://www.rightscale.com/blog/cloud-industry-insights/googl...
- cordite 12y agoI've only seen this feature used when hosts are being upgraded at the hypervisor or physical level. In my experience, it has been more of a "keep the whole system stable and running during upgrades" rather than a daily optimization strategy.
- lifty 12y agoContainers can also do live migrations. Check out the CRIU project(http://criu.org/ http://criu.org/)
- ChuckMcM 12y agoInteresting article mostly because it tells me that Containers are clearly getting traction (Citrix sees them as a threat to their business, so they declare they are not a threat to their business.) That said, there are much deeper questions here than 'container good or bad?' Not surprisingly, they are nearly the same as 'compilers good or bad?' that were all the rage before. So hypervisors and containers are the same two systems, hypervisors abstract to the hardware level and containers abstract to the OS level. By setting the level of abstraction they control what can be shared and what can't, and what differences can be accommodated in the same system. At scale, efficiency is associative. Which is to say that you can be more efficient in your use of resources (higher utilization) when you associate similar abstractions together. You do that by maximizing the commonality. So if the only common thing is 'runs on x86 hardware' that is your common point. If its 'runs on Linux' (or Windows, or Solaris) as a common point then that you get some efficiency by running one copy of the OS and multiple processes in that OS. If the common point is 'Rails' or 'Go' then you can gain efficency by keeping hardware, OS, and software stack in common. The further up the abstraction level you go, it loosens constraints on the bottom layers while increases them on the top layers (you have to write all your code in Rails, but you can run on x86-64 or ARM-64 architecture machines, versus you can write you code in any language/system that targets x86-64 hardware) Different organizations can dial that knob based on their ability to be flexible. So organizations with a lot of legacy binaries targeting x86-64 they really need hypervisors, but organizations where all of their code base targets Linux, they are much better off with containers. At a small scale the numbers are a toss up, at Google/Walmart scale the numbers become company controlling.
- jgrodziski 12y agoI think Unikernels are the next steps after the lightweight containers trend, in short an execution environment free of all the unused clutter an OS brings for just executing one application. http://t.co/JRxG2RGRgp http://t.co/JRxG2RGRgp What do you think of this?
- amirmc 12y agoI completely agree and the workflow does away with the need for a vast number of configuration management tools too. Also, John Carmack also thinks they're "really interesting" [1] (the link in his tweet takes you to the academic paper). If you've used Unikernels for anything, I'd really like to know about your experience. Please do get in touch with me (see profile). Side note: Why use a Twitter short URL when the original [2] is pretty succinct anyway? [1] https://twitter.com/ID_AA_Carmack/status/476046680698003458 https://twitter.com/ID_AA_Carmack/status/476046680698003458 [2] http://queue.acm.org/detail.cfm?id=2566628 http://queue.acm.org/detail.cfm?id=2566628
- jgrodziski 12y agoHi Amir, Thanks for your reply, don't have time to invest that much in unikernel, I just keep an eye on it. But the use case is very appealing to me and to you according to your writings on your blog :) I'll keep in touch if I investigate further the Unikernel way. Thanks for the John Carmack's tweet, very interesting, and a validation of our intuition by one of the greatest programmers :) Side note : I sent the comment from my smartphone and the quickest way to retrieve the link to the ACM article on Unikernel is by browsing my twitter feed :) hence the twitter short URL
- krakensden 12y agoI'm pretty skeptical. First, it's rarely ever really one application. There's a primary application, sure, but there's also a bunch of processes that monitor it, clean up its logs, perform system maintenance, etc. Second, you lose 100% of the tooling for debugging. Finally, and most importantly, your operating system does an awful lot of stuff that is not trivial, not in the slightest. The Linux kernel has better page cache algorithms than your unikernel, a better networking stack, vastly better hardware support, etc. Plus tons of people thinking about how to make it better every day. No one is going to port that to OCaml.
- atc 12y agoObviously the blog's owner has a vested interest. That said, there are plenty of use cases where containers are perfect because kernel-level issues aren't a problem: RDBMS hosts, web/app containers. Lots of "enterprisey" stuff that isn't kernel-level dependent (beyond that of its dependencies). Let's stop looking at things as "silver bullets" and accept containers for what they are: very useful in lots of use cases, but virtualisation is more appropriate in others. Choice is great!
- thejdude 12y agoIf you're one of the 0.01% whose app cannot run on Linux, or needs a very specific patched version of the kernel to run, then sure, I agree. Who knows, maybe Citrix has something to sell that makes the author's opinion very "important"?
- dschiptsov 12y agoLet's say that unless you are mega-provider, which trying to pack as many as possible those cheap low-resources (10 pages, 3 forms, 5 tables, 10 requests per day Rails or whatever is current buzzword) site into one piece of leased hardware the visualization is a wrong question. Just apache's virtual hosts isn't good enough anymore, customers want Rails, Nodejs, Java for their $20/mo. Virtualization for real-world production is nonsense, especially those idiotic setups like Java under Linux under VMWare under Windows (you wouldn't believe how many "professionals" are cock-sure that this kind of setup is "robust" and most stable.) Jokes aside, Docker is eating VMWare's lunch being a fresh new meme promising miracles (yeah, problem of cache trashing and context switching for I/O in a host OS will surely be solved by someone else.
- deleted 12y ago[deleted]