8 ms·

Hyper Is Docker Done the Right Way

98 points by mrmrcoleman 10y ago

FLGMwt 10y ago
Is this not what ECS aims to do?
- mrmrcoleman 10y ago
  The difference between Hyper and ECS is that ECS (and Docker Cloud, and GCE) all require that you provision and manage your own Docker server cluster. With Hyper this is abstracted away which has a few benefits. Hyper also runs on HyperContainers (https://docs.hypercontainer.io/ https://docs.hypercontainer.io/) which provides secure (VM like) multi-tenancy with container agility.
  - josegonzalez 10y ago
    Excuse me for my ignorance, but what does "container agility" mean?
    - initdaemon 10y ago
      He was talking about the boot speed, and the push/pull image workflow.
  - michaelsbradley 10y ago
    But at some point doesn't one need to think at the granularity level of clusters? In other words, there are natural limits to how big the hosting environment can grow, at which point performance and efficiency with respect to network topology and other host-system factors (in the broadest senses, e.g. at the level of data centers and the tech-pieces from which they're built) must be considered. For reasons of convenience, in the case of some (many?) applications, perhaps those considerations can be ignored and the "container fabric" can simply be thought of as perfectly homogeneous, is that the idea? And if one grows beyond the point where that simplifying assumption doesn't hold, then it's time to switch to an architecture where clustering is explicitly addressed?
    - initdaemon 10y ago
      In that sense, shall we care whether the EC2 instances in the ECS cluster are located at the same server, or the same rack?
      - michaelsbradley 10y ago
        It would depend on "how things are wired together". Now, it's great if they are "wired together" in such a way that one need not give it any thought. But whether we're talking EC2 instances or containers, at some point one has to think about it, e.g. two instances talking to each other, one being on the East Coast USA and the other in the Midwest, or West Coast, vs. their both being in the same datacenter. That's at the extremes, for sure, but maybe even intra-dc clustering has to be considered explicitly for certain applications? Maybe not?
        initdaemon 10y ago
        The point is that you should stop thinking about VM cluster, but container (application) cluster, aka microservices.
    - mrmrcoleman 10y ago
      Broadly yes. You could apply a similar argument to Digital Ocean or other providers. It is feasible that you could outgrow any provider, but in practice they would be able to scale before you hit that problem. Is that what you meant or did I misunderstand?
teilo 10y ago
Other than using the Docker format / API, how is this not just a variant VM deployment strategy? If every container is running its own kernel on a hypervisor, doesn't this eliminate one of the key benefits of a true Container system, the kernel/memory overhead?
- scprodigy 10y ago
  What overhead?
  - TickleSteve 10y ago
    The benefit of a container over a full VM is meant to be that the container scheme has just one kernel running. This scheme would effectively be just one container per VM, thus negating the benefit. (Edit: So.. it would appear that the benefit is that they're read-only-sharing the kernel between VMs)
    - scprodigy 10y ago
      In terms of a public cloud service, you (customer) don't care. You pay for the amount of memory you used, whether there is one kernel, or many of them, is irrelevant.
      - teilo 10y ago
        It is absolutely relevant: If your container is running its own kernel, then it is consuming memory for that kernel, not to mention the CPU overhead of the hosted kernel. Additionally, every container must also boot a complete OS. More overhead. This is why traditional containers use a shared kernel with process group isolation. You pay for what you use. A traditional container only uses what it needs for the app itself, and it starts in a fraction of the time because it doesn't have to boot anything. Performance and cost may be acceptable regardless, but that's not the point. Containers are more efficient. Based upon responses from Hyper, they appear to address these concerns in a manner I have never heard before. I will certainly be looking into their core technology.
neikos 10y ago
> "CoreOS, DigitalOcean, and Docker are sponsors of The New Stack." Interesting and somewhat funny
- bergwolf 10y ago
  Another funny thing is that Docker just got slapped in the face "Docker in Production: A History of Failure" (https://news.ycombinator.com/item?id=12872304 https://news.ycombinator.com/item?id=12872304).
  - daenney 10y ago
    I wouldn't consider that a slap in the face. It's a rant, it's not the first about Docker and it won't be the last. Docker seems to be doing just fine despite that and many posts like it. Though the author has some good points it's riddled with statements that range from dubious to factually false.
eggie5 10y ago
His final comments on K8s & GKE were the same conclusion I came to while migrating my personal website to k8s: http://www.eggie5.com/82-rails-docker-app-deployment-kubernetes http://www.eggie5.com/82-rails-docker-app-deployment-kuberne...
- mrmrcoleman 10y ago
  Hey eggie5, you mean the point he makes about scale?
- scprodigy 10y ago
  The value proposition (difference) of Hyper is that while other CaaS try to manage your VM cluster, Hyper makes the cluster just go away.
aplanas 10y ago
I like the initial idea very much. As I can see this is a wrapper over QEMU and controlled by QMP [1][2], so they are providing a kernel, initrd and a image. It is also configuring a machine profile (memory, vcpu, bus, devices, etc.). There are also interfaces for libvirt, xen and others virtualization technologies. This also looks similar of what Docker is doing now in Windows with HyperV. My question here is: from where comes the speed improvement in relation with a classical VM approach? [1] https://github.com/hyperhq/runv/blob/master/hypervisor/qemu/qemu_amd64.go#L84 https://github.com/hyperhq/runv/blob/master/hypervisor/qemu/... [2] https://github.com/hyperhq/runv/blob/master/hypervisor/qemu/qemu_process.go#L135 https://github.com/hyperhq/runv/blob/master/hypervisor/qemu/...
- gnawux 10y ago
  The most significant improvement comes from the BootFromTemplate, which improves speed and saves memory. And the boot sequence and guest kernel have been optimized.
  - antocv 10y ago
    How has the guest kernel been optimized? Just recompiled vanilla kernel and stripped all unnecessary parts? What do you use for storage, how is the hosts storage accessed from the guest-container? virtfs? Doesnt that add a bottleneck/performance-problem, even if youre using virtfs thats several layers of translation from VFS/block-device-virtual(what kind do you use?)/PCI-bus to VFS/block-device/pci and so on.
- colemickens 10y ago
  How is this different than just using rkt with the kvm stage1? (which to my understanding does basically the same thing)
AcerbicZero 10y ago
Maybe I'm late to the party, but this looks almost exactly like what VMware did last year, with their vSphere Integrated Containers.
- gnawux 10y ago
  VMware has a similar project. The HyperContainer project is opensourced in May last year, same as Intel clear linux, and is prior to the VMware project.

98 points by mrmrcoleman 10y ago

FLGMwt 10y ago

Is this not what ECS aims to do?

mrmrcoleman 10y ago
The difference between Hyper and ECS is that ECS (and Docker Cloud, and GCE) all require that you provision and manage your own Docker server cluster. With Hyper this is abstracted away which has a few benefits. Hyper also runs on HyperContainers (https://docs.hypercontainer.io/ https://docs.hypercontainer.io/) which provides secure (VM like) multi-tenancy with container agility.
- josegonzalez 10y ago
  Excuse me for my ignorance, but what does "container agility" mean?
  - initdaemon 10y ago
    He was talking about the boot speed, and the push/pull image workflow.
- michaelsbradley 10y ago
  But at some point doesn't one need to think at the granularity level of clusters? In other words, there are natural limits to how big the hosting environment can grow, at which point performance and efficiency with respect to network topology and other host-system factors (in the broadest senses, e.g. at the level of data centers and the tech-pieces from which they're built) must be considered. For reasons of convenience, in the case of some (many?) applications, perhaps those considerations can be ignored and the "container fabric" can simply be thought of as perfectly homogeneous, is that the idea? And if one grows beyond the point where that simplifying assumption doesn't hold, then it's time to switch to an architecture where clustering is explicitly addressed?
  - initdaemon 10y ago
    In that sense, shall we care whether the EC2 instances in the ECS cluster are located at the same server, or the same rack?
    - michaelsbradley 10y ago
      It would depend on "how things are wired together". Now, it's great if they are "wired together" in such a way that one need not give it any thought. But whether we're talking EC2 instances or containers, at some point one has to think about it, e.g. two instances talking to each other, one being on the East Coast USA and the other in the Midwest, or West Coast, vs. their both being in the same datacenter. That's at the extremes, for sure, but maybe even intra-dc clustering has to be considered explicitly for certain applications? Maybe not?
      - initdaemon 10y ago
        The point is that you should stop thinking about VM cluster, but container (application) cluster, aka microservices.
  - mrmrcoleman 10y ago
    Broadly yes. You could apply a similar argument to Digital Ocean or other providers. It is feasible that you could outgrow any provider, but in practice they would be able to scale before you hit that problem. Is that what you meant or did I misunderstand?

teilo 10y ago

Other than using the Docker format / API, how is this not just a variant VM deployment strategy? If every container is running its own kernel on a hypervisor, doesn't this eliminate one of the key benefits of a true Container system, the kernel/memory overhead?

scprodigy 10y ago
What overhead?
- TickleSteve 10y ago
  The benefit of a container over a full VM is meant to be that the container scheme has just one kernel running. This scheme would effectively be just one container per VM, thus negating the benefit. (Edit: So.. it would appear that the benefit is that they're read-only-sharing the kernel between VMs)
  - scprodigy 10y ago
    In terms of a public cloud service, you (customer) don't care. You pay for the amount of memory you used, whether there is one kernel, or many of them, is irrelevant.
    - teilo 10y ago
      It is absolutely relevant: If your container is running its own kernel, then it is consuming memory for that kernel, not to mention the CPU overhead of the hosted kernel. Additionally, every container must also boot a complete OS. More overhead. This is why traditional containers use a shared kernel with process group isolation. You pay for what you use. A traditional container only uses what it needs for the app itself, and it starts in a fraction of the time because it doesn't have to boot anything. Performance and cost may be acceptable regardless, but that's not the point. Containers are more efficient. Based upon responses from Hyper, they appear to address these concerns in a manner I have never heard before. I will certainly be looking into their core technology.

neikos 10y ago

> "CoreOS, DigitalOcean, and Docker are sponsors of The New Stack." Interesting and somewhat funny

bergwolf 10y ago
Another funny thing is that Docker just got slapped in the face "Docker in Production: A History of Failure" (https://news.ycombinator.com/item?id=12872304 https://news.ycombinator.com/item?id=12872304).
- daenney 10y ago
  I wouldn't consider that a slap in the face. It's a rant, it's not the first about Docker and it won't be the last. Docker seems to be doing just fine despite that and many posts like it. Though the author has some good points it's riddled with statements that range from dubious to factually false.

eggie5 10y ago

His final comments on K8s & GKE were the same conclusion I came to while migrating my personal website to k8s: http://www.eggie5.com/82-rails-docker-app-deployment-kubernetes http://www.eggie5.com/82-rails-docker-app-deployment-kuberne...

mrmrcoleman 10y ago
Hey eggie5, you mean the point he makes about scale?
scprodigy 10y ago
The value proposition (difference) of Hyper is that while other CaaS try to manage your VM cluster, Hyper makes the cluster just go away.

aplanas 10y ago

I like the initial idea very much. As I can see this is a wrapper over QEMU and controlled by QMP [1][2], so they are providing a kernel, initrd and a image. It is also configuring a machine profile (memory, vcpu, bus, devices, etc.). There are also interfaces for libvirt, xen and others virtualization technologies. This also looks similar of what Docker is doing now in Windows with HyperV. My question here is: from where comes the speed improvement in relation with a classical VM approach? [1] https://github.com/hyperhq/runv/blob/master/hypervisor/qemu/qemu_amd64.go#L84 https://github.com/hyperhq/runv/blob/master/hypervisor/qemu/... [2] https://github.com/hyperhq/runv/blob/master/hypervisor/qemu/qemu_process.go#L135 https://github.com/hyperhq/runv/blob/master/hypervisor/qemu/...

gnawux 10y ago
The most significant improvement comes from the BootFromTemplate, which improves speed and saves memory. And the boot sequence and guest kernel have been optimized.
- antocv 10y ago
  How has the guest kernel been optimized? Just recompiled vanilla kernel and stripped all unnecessary parts? What do you use for storage, how is the hosts storage accessed from the guest-container? virtfs? Doesnt that add a bottleneck/performance-problem, even if youre using virtfs thats several layers of translation from VFS/block-device-virtual(what kind do you use?)/PCI-bus to VFS/block-device/pci and so on.
colemickens 10y ago
How is this different than just using rkt with the kvm stage1? (which to my understanding does basically the same thing)

AcerbicZero 10y ago

Maybe I'm late to the party, but this looks almost exactly like what VMware did last year, with their vSphere Integrated Containers.

gnawux 10y ago
VMware has a similar project. The HyperContainer project is opensourced in May last year, same as Intel clear linux, and is prior to the VMware project.