5 ms·

Networking is a disaster. Kernel level networking is nice, but that requires access to the kernel which you can't provide in a container. Doing so means contain

by api 12y ago

Networking is a disaster. Kernel level networking is nice, but that requires access to the kernel which you can't provide in a container. Doing so means containers no longer contain. IPV6 was supposed to solve a lot of this by having an address space so huge you could easily give every vm host a few billion IPs. But nobody uses it.

ploxiln 12y ago
Containers already fully depend on a shared and complicated kernel to do containment (which is why some say "containers don't contain" e.g. http://www.projectatomic.io/blog/2014/09/yet-another-reason-containers-don-t-contain-kernel-keyrings/ http://www.projectatomic.io/blog/2014/09/yet-another-reason-... ). Virtual Machines have a smaller attack interface to the host, so they do have a chance of containing malicious code. Kernels were supposed to contain processes, providing them with a virtual private memory space, virtual storage access so they can't read or write any file you don't want them to, etc... but only openbsd is really trying to make that work.
thinkingkong 12y ago
Networking isnt a disaster. It's just that the current container ecosystem on linux hasnt yet resulted in any real domain specific improvements yet. For "default" systems, networking is pretty damned reliable and performant. A lot of the problems with regards to containment have already been solved in different systems. I believe solaris or opensolaris had Crossbow [1]. Any system that aims to provide connectivity needs to do the least amount of encapsulation possible and probably be a kernel module. 1. http://en.m.wikipedia.org/wiki/OpenSolaris_Network_Virtualization_and_Resource_Control http://en.m.wikipedia.org/wiki/OpenSolaris_Network_Virtualiz...
- adamc 12y ago
  "the least amount of encapsulation possible" doesn't sound like much of a solution if what you are looking for is networking with strong encapsulation.
  - hueving 12y ago
    >strong encapsulation What does that term even mean? Are you talking about encryption? If not, there is no 'strength' to encapsulation. Something is either encapsulated efficiently or it's not.
    - adamc 12y ago
      That's silly. There are definitely degrees of encapsulation.
- bcantrill 12y ago
  For whatever it's worth, the Crossbow technology remains at the core of our Triton container stack[1] -- and we have extended it significantly by adding VXLAN and virtual layer 2 networking.[2][3] One of the comments we have heard from many early Triton adopters is that they enjoy having IP addresses, networking stacks and bare-metal performance. ;) [1] https://www.joyent.com/blog/triton-docker-and-the-best-of-all-worlds https://www.joyent.com/blog/triton-docker-and-the-best-of-al... [2] http://fingolfin.org/illumos/talks/sdn.pdf http://fingolfin.org/illumos/talks/sdn.pdf [3] https://www.youtube.com/watch?v=mLvi3nPNqiQ&list=PLH8r-Scm3-2VmZhZ76tFPAhPOG0pvmjdA&index=3 https://www.youtube.com/watch?v=mLvi3nPNqiQ&list=PLH8r-Scm3-...
justincormack 12y ago
Hey some of us believe that ipv6 is a good solution for containers. There are even products that support it (like Calico http://calicoproject.org/ http://calicoproject.org/ ) and even a few cloud providers (not digital ocean who think that 16 ipv6 addresses is ipv6). If you want a real ip/ipv6 stack for your container, try unikernels.
- swansonc 12y ago
  Thank's for the IPv6 love on Project Calico, Justin. Have you been testing Calico's v6? If so, we'd love to talk to you (disclosure, I'm on the project calico team).
  - justincormack 12y ago
    Not yet, planning to give it a go soon...
    - swansonc 12y ago
      drop me a line when you do. Containers or OpenStack? cdl (at) projectcalico
- eepyaich 12y ago
  Justin, I think/hope you meant http://projectcalico.org/ http://projectcalico.org/!
sophacles 12y ago
How the heck are you running containers without kernel access? Does your software not malloc or read files or you know... execute?
- api 12y ago
  Containers implement something not unlike SELinux policies. Networking stuff requires the NET_ADMIN permission, which is not set by default.
  - wmf 12y ago
    Setting up networking requires root but so does creating the container so I don't see what the problem is. Using networking doesn't require any special permissions.
aristotle 12y ago
It is not a disaster. Its possible to build a performant and easy networking solution for containers. Except the vendors are in a hurry to get some hacky software out that takes back the clock 20 years.

by api 12y ago

ploxiln 12y ago

Containers already fully depend on a shared and complicated kernel to do containment (which is why some say "containers don't contain" e.g. http://www.projectatomic.io/blog/2014/09/yet-another-reason-containers-don-t-contain-kernel-keyrings/ http://www.projectatomic.io/blog/2014/09/yet-another-reason-... ). Virtual Machines have a smaller attack interface to the host, so they do have a chance of containing malicious code. Kernels were supposed to contain processes, providing them with a virtual private memory space, virtual storage access so they can't read or write any file you don't want them to, etc... but only openbsd is really trying to make that work.

thinkingkong 12y ago

Networking isnt a disaster. It's just that the current container ecosystem on linux hasnt yet resulted in any real domain specific improvements yet. For "default" systems, networking is pretty damned reliable and performant. A lot of the problems with regards to containment have already been solved in different systems. I believe solaris or opensolaris had Crossbow [1]. Any system that aims to provide connectivity needs to do the least amount of encapsulation possible and probably be a kernel module. 1. http://en.m.wikipedia.org/wiki/OpenSolaris_Network_Virtualization_and_Resource_Control http://en.m.wikipedia.org/wiki/OpenSolaris_Network_Virtualiz...

adamc 12y ago
"the least amount of encapsulation possible" doesn't sound like much of a solution if what you are looking for is networking with strong encapsulation.
- hueving 12y ago
  >strong encapsulation What does that term even mean? Are you talking about encryption? If not, there is no 'strength' to encapsulation. Something is either encapsulated efficiently or it's not.
  - adamc 12y ago
    That's silly. There are definitely degrees of encapsulation.
bcantrill 12y ago
For whatever it's worth, the Crossbow technology remains at the core of our Triton container stack[1] -- and we have extended it significantly by adding VXLAN and virtual layer 2 networking.[2][3] One of the comments we have heard from many early Triton adopters is that they enjoy having IP addresses, networking stacks and bare-metal performance. ;) [1] https://www.joyent.com/blog/triton-docker-and-the-best-of-all-worlds https://www.joyent.com/blog/triton-docker-and-the-best-of-al... [2] http://fingolfin.org/illumos/talks/sdn.pdf http://fingolfin.org/illumos/talks/sdn.pdf [3] https://www.youtube.com/watch?v=mLvi3nPNqiQ&list=PLH8r-Scm3-2VmZhZ76tFPAhPOG0pvmjdA&index=3 https://www.youtube.com/watch?v=mLvi3nPNqiQ&list=PLH8r-Scm3-...

justincormack 12y ago

Hey some of us believe that ipv6 is a good solution for containers. There are even products that support it (like Calico http://calicoproject.org/ http://calicoproject.org/ ) and even a few cloud providers (not digital ocean who think that 16 ipv6 addresses is ipv6). If you want a real ip/ipv6 stack for your container, try unikernels.

swansonc 12y ago
Thank's for the IPv6 love on Project Calico, Justin. Have you been testing Calico's v6? If so, we'd love to talk to you (disclosure, I'm on the project calico team).
- justincormack 12y ago
  Not yet, planning to give it a go soon...
  - swansonc 12y ago
    drop me a line when you do. Containers or OpenStack? cdl (at) projectcalico
eepyaich 12y ago
Justin, I think/hope you meant http://projectcalico.org/ http://projectcalico.org/!

sophacles 12y ago

How the heck are you running containers without kernel access? Does your software not malloc or read files or you know... execute?

api 12y ago
Containers implement something not unlike SELinux policies. Networking stuff requires the NET_ADMIN permission, which is not set by default.
- wmf 12y ago
  Setting up networking requires root but so does creating the container so I don't see what the problem is. Using networking doesn't require any special permissions.

aristotle 12y ago

It is not a disaster. Its possible to build a performant and easy networking solution for containers. Except the vendors are in a hurry to get some hacky software out that takes back the clock 20 years.