4 ms·

Comparing Triton containers to VMs and bare metal servers

43 points by xj9 11y ago

deleted 11y ago
[deleted]
gtirloni 11y ago
Containers hosting hardware virtual machines running Windows, FreeBSD, or others Can't we just describe VMs as VMs and be done with it? So it's a container that has a VM inside it?
- bcantrill 11y ago
  No, it's not -- a Triton container is a SmartOS container running an LX-branded zone.[1][2] [1] http://www.slideshare.net/bcantrill/illumos-lx http://www.slideshare.net/bcantrill/illumos-lx [2] https://www.joyent.com/blog/triton-docker-and-the-best-of-all-worlds https://www.joyent.com/blog/triton-docker-and-the-best-of-al... Edit: I think I misunderstood the question -- apologies!
- jclulow 11y ago
  Yes, the QEMU process that underpins the hardware virtual machine lives within a container on the host. It means that a vulnerability in QEMU that allows for guest escape will land you in an otherwise empty container, rather than out on the host where other tenants are accessible. But Triton is first and foremost about _container-based_ virtualisation: which involves no HVM at all!
  - gtirloni 11y ago
    Ah, that's something I hadn't considered. Good point. I guess the overhead will be minimum in that scenario.
  - xenophonf 11y ago
    Is it possible to do the same thing with cgroups or jails?
johannesboyne 11y ago
https://www.youtube.com/watch?v=Ll50EFquwSo https://www.youtube.com/watch?v=Ll50EFquwSo Building and scaling container driven cloud infrastructure - Bryan Cantrill (Video about Triton)
logicallee 11y ago
there are about 60 mentions of the word "container" on that page, including a full chart of questions "Compared to Docker in a VM", such as "Where are containers run"? But the real question that should be at the top of the file, or at least that table of questions, is "what is a container"? So: what is a container? (As opposed to a VM.)
- dmpk2k 11y ago
  Container is a chroot jail on steroids. VM is a QEMU process (also in a container).
  - logicallee 11y ago
    how can you run a totally different architecture from just a chroot jail? It doesn't make sense. How are you "on the metal" if you're running some totally different kernel from what the chroot-jail-host is running? You can't both be running the kernel on the metal...
    - dmpk2k 11y ago
      I'm not clear on the question. The containers don't run a different kernel. They're just regular processes, in an fancy evolution of a chroot jail. If you want to run something like Windows or FreeBSD, then you break out QEMU.
    - bcantrill 11y ago
      It might help to see my presentation entitled "Docker and the future of containers in production"[1][2]; it covers the fundamental building blocks here, and should answer your questions. (If that presentation leaves you looking for more depth on the LX branded zone technology on SmartOS, see my Surge illumos day presentation from last fall.[3]) [1] https://www.joyent.com/developers/videos/docker-and-the-future-of-containers-in-production https://www.joyent.com/developers/videos/docker-and-the-futu... [2] http://www.slideshare.net/bcantrill/docker-and-the-future-of-containers-in-production http://www.slideshare.net/bcantrill/docker-and-the-future-of... [3] http://www.slideshare.net/bcantrill/illumos-lx http://www.slideshare.net/bcantrill/illumos-lx
robbiet480 11y ago
Surprising that Joyent used an extremely similar name as Amazon's extremely similar product, EC2 Container Service (EC2 expands to Elastic Compute Cloud, so the full name is Elastic Compute Cloud Container Service) http://aws.amazon.com/ecs/ http://aws.amazon.com/ecs/
- omarforgotpwd 11y ago
  The name is very similar but the products are not so similar -- ECS simply schedules Docker containers across the virtual machines you already have running on AWS. Triton schedules and runs containers on bare metal. Virtualization of course has performance and I/O overhead.
kstenerud 11y ago
Now if only they fully supported AMD CPUs...
- bahamat 11y ago
  SmartOS runs just fine on AMD CPUs. KVM support for AMD hasn't been merged to illumos-kvm yet (mostly just due to time), but you can do your own builds with it (https://github.com/jclulow/illumos-kvm/ https://github.com/jclulow/illumos-kvm/). But Triton containers don't run in KVM anyway. They run natively on the OS with no VM. So you can download and run Triton on AMD today, just as you can with Intel CPUs.
mwcampbell 11y ago
I've been watching Joyent with interest for a while now, but I don't think I or my employer will be using their public cloud any time soon. Frankly, the pricing standard set by DigitalOcean a couple of years ago, and since followed by Linode (my and my employer's current favorite) and Vultr, is pretty hard to beat. I don't know how I'd justify 2x the price (not even counting bandwidth) when the performance of Linode is good enough. So I'll just continue to watch with interest. Now that SmartDataCenter is open source, maybe someone else will set up an SDC cloud with pricing that's competitive with DigitalOcean and the like. That would suck for Joyent though, so I shouldn't really wish for it.

43 points by xj9 11y ago

deleted 11y ago

[deleted]

Containers hosting hardware virtual machines running Windows, FreeBSD, or others Can't we just describe VMs as VMs and be done with it? So it's a container that has a VM inside it?

bcantrill 11y ago
No, it's not -- a Triton container is a SmartOS container running an LX-branded zone.[1][2] [1] http://www.slideshare.net/bcantrill/illumos-lx http://www.slideshare.net/bcantrill/illumos-lx [2] https://www.joyent.com/blog/triton-docker-and-the-best-of-all-worlds https://www.joyent.com/blog/triton-docker-and-the-best-of-al... Edit: I think I misunderstood the question -- apologies!
jclulow 11y ago
Yes, the QEMU process that underpins the hardware virtual machine lives within a container on the host. It means that a vulnerability in QEMU that allows for guest escape will land you in an otherwise empty container, rather than out on the host where other tenants are accessible. But Triton is first and foremost about _container-based_ virtualisation: which involves no HVM at all!
- gtirloni 11y ago
  Ah, that's something I hadn't considered. Good point. I guess the overhead will be minimum in that scenario.
- xenophonf 11y ago
  Is it possible to do the same thing with cgroups or jails?

johannesboyne 11y ago

https://www.youtube.com/watch?v=Ll50EFquwSo https://www.youtube.com/watch?v=Ll50EFquwSo Building and scaling container driven cloud infrastructure - Bryan Cantrill (Video about Triton)

logicallee 11y ago

there are about 60 mentions of the word "container" on that page, including a full chart of questions "Compared to Docker in a VM", such as "Where are containers run"? But the real question that should be at the top of the file, or at least that table of questions, is "what is a container"? So: what is a container? (As opposed to a VM.)

dmpk2k 11y ago
Container is a chroot jail on steroids. VM is a QEMU process (also in a container).
- logicallee 11y ago
  how can you run a totally different architecture from just a chroot jail? It doesn't make sense. How are you "on the metal" if you're running some totally different kernel from what the chroot-jail-host is running? You can't both be running the kernel on the metal...
  - dmpk2k 11y ago
    I'm not clear on the question. The containers don't run a different kernel. They're just regular processes, in an fancy evolution of a chroot jail. If you want to run something like Windows or FreeBSD, then you break out QEMU.
  - bcantrill 11y ago
    It might help to see my presentation entitled "Docker and the future of containers in production"[1][2]; it covers the fundamental building blocks here, and should answer your questions. (If that presentation leaves you looking for more depth on the LX branded zone technology on SmartOS, see my Surge illumos day presentation from last fall.[3]) [1] https://www.joyent.com/developers/videos/docker-and-the-future-of-containers-in-production https://www.joyent.com/developers/videos/docker-and-the-futu... [2] http://www.slideshare.net/bcantrill/docker-and-the-future-of-containers-in-production http://www.slideshare.net/bcantrill/docker-and-the-future-of... [3] http://www.slideshare.net/bcantrill/illumos-lx http://www.slideshare.net/bcantrill/illumos-lx

robbiet480 11y ago

Surprising that Joyent used an extremely similar name as Amazon's extremely similar product, EC2 Container Service (EC2 expands to Elastic Compute Cloud, so the full name is Elastic Compute Cloud Container Service) http://aws.amazon.com/ecs/ http://aws.amazon.com/ecs/

omarforgotpwd 11y ago
The name is very similar but the products are not so similar -- ECS simply schedules Docker containers across the virtual machines you already have running on AWS. Triton schedules and runs containers on bare metal. Virtualization of course has performance and I/O overhead.

kstenerud 11y ago

Now if only they fully supported AMD CPUs...

bahamat 11y ago
SmartOS runs just fine on AMD CPUs. KVM support for AMD hasn't been merged to illumos-kvm yet (mostly just due to time), but you can do your own builds with it (https://github.com/jclulow/illumos-kvm/ https://github.com/jclulow/illumos-kvm/). But Triton containers don't run in KVM anyway. They run natively on the OS with no VM. So you can download and run Triton on AMD today, just as you can with Intel CPUs.

mwcampbell 11y ago

I've been watching Joyent with interest for a while now, but I don't think I or my employer will be using their public cloud any time soon. Frankly, the pricing standard set by DigitalOcean a couple of years ago, and since followed by Linode (my and my employer's current favorite) and Vultr, is pretty hard to beat. I don't know how I'd justify 2x the price (not even counting bandwidth) when the performance of Linode is good enough. So I'll just continue to watch with interest. Now that SmartDataCenter is open source, maybe someone else will set up an SDC cloud with pricing that's competitive with DigitalOcean and the like. That would suck for Joyent though, so I shouldn't really wish for it.