4 ms·

Don't get me wrong, I'm a huge fan of LXC and I've collaborated with the LXC folks with upstream kernel work (they're really clever guys). > The important part

• hardwaresofton 8y ago
  I've also struggled with the distinction, but I believe the commenter above was referring to LXC (and LXD's) "system containers". Full user namespacing has been in from very early on (day 1?) which is the "increased security" -- this is where I believed the difference between "system containers" for LXC and regular "application containers" was. runc was not always the runtime underneath Docker, and IIRC it has not always had support for proper user namespacing for as long as LXC has. Also, LXC has so many more features packed in I think it's a disservice to compare it to runc in the first place. It should go without saying but I would absolutely love for you to sprinkle some knowledge on me wherever I'm wrong about either of these. [EDIT] - Getting downvotes so figured maybe I said something egregious so I did some digging: - LXC had user namespaces @ v1.0 in 2013[0] - Docker got user namespaces @ 1.10 in 2016[1] This is the distinction I meant. [0]: https://stgraber.org/2014/01/01/lxc-1-0-security-features/ https://stgraber.org/2014/01/01/lxc-1-0-security-features/ [1]: https://blog.docker.com/2016/02/docker-1-10/ https://blog.docker.com/2016/02/docker-1-10/
  • cyphar 8y ago
    > Full user namespacing has been in from very early on (day 1?) I'm not sure this is true. User namespaces were added a while after LXC first started development (from memory), and weren't safe (at all) for several years. There was a lot of churn to make user namespaces work (and both LXC and runc had issues with them). It should be noted that the user namespacing available in LXC is emphatically the correct way of doing it, and few other runtimes have had similarly correct approach. rkt had a similar idea, but never really went all the way with the extra work. Christian Brauner (one of the LXC maintainers) did quite a bit of work to extend the mapping limit for user namespaces just so that LXC could do even more with them. runc _could_ be used in a similar way to LXC (after all, fro userspace user namespaces are just three files and a clone flag), but you'd need to do the management separately. And doing it right is quite hard when you get in the weeds -- and I absolutely give props to the LXC folks for getting them right. > IIRC it has not always had support for proper user namespacing for as long as LXC has. runc has had user namespace support for a shorter time because the project is younger. User namespaces aren't particularly difficult to use (though a lot of the anciliary work is quite significant -- and LXC does a lot of things that are outside the scope of runc). Most of the delay for Docker to have user namespaces was purely because of Docker (and I even think runc wasn't a separate project when Docker got user namespace support -- it was still libcontainer back then). > LXC has so many more features packed in I think it's a disservice to compare it to runc in the first place. I absolutely agree LXC has many more features (many of them that I'm jealous of), it's just that runc is the most apt comparison to it from the Docker stack. I disagree it's a disservice, given that I know the LXC folks personally and they feel it's okay to make such comparisons (especially when we're discussing individual runtime issues that both runtimes had to deal with in different ways). I'd never claim that runc does things much better than LXC -- it has lots of really great engineering that I wish more people took advantage of. There are only two things that runc has over LXC: * In our design, there are no monitor or long-running processes. The upside of this is that you don't have to worry about us being bad at programming -- your container is your business. The downside is that you likely will need to manage your containers somehow (getting the exit code is the most obvious thing you currently need a monitor process for). And console handling is quite complicated. But it does allow for quite a bit of flexibility in the layers above runc -- and it took quite a while to get this right. * runc has a fully-unprivileged mode ("rootless containers"). Now, LXC can do most of this, but with runc my main idea was that we needed to have completely unprivileged containers (no admin setup, no setuid helpers, and so on). This has pioneered a few projects (slirp4netns -- a highly performant unprivileged user-space veth bridge) which actually now allow you to do an incredibly large number of things without any privileges at any step in the process. So while LXC could do most of this, it used several setuid binaries which made it not work for me -- and runc's rootless container support has really made a lot of strides that wouldn't have been made without the work I did in runc. But those are fairly minor gripes. LXC even has OCI image support with templates (using a tool I wrote called "umoci" :P), and there were plans to add OCI runtime configuration support as well.
• tobbyb 8y ago
  Docker was based on LXC and there are some important differences that should be more well known. The main difference is LXC boots an init in the namespace as the first process so you get a standard OS environment with multi-process support. Docker runs the application directly so you get a single process environment, unless you decide to run a process manager. IMHO this creates far more problems than it solves as anything nonstandard would, as you now need workarounds for basics like logging, networking, daemons and anything that expects a standard OS environment Docker also uses layers to build containers and ephemeral data. None of these 3 critical decisions have seen much technical scrutiny resulting in persistent confusion in the ecosystem and with scrutiny many users will find they can add far more management overhead and complexity to containers. We have written about some of these issues here. [1] For instance OP is referring to unprivileged containers and user namespaces that let you run containers as a user process. LXC has supported these since 2013. Docker added user namespaces support in recent versions. However taking just one issue - because of the use of layers to build containers you can't use unprivileged containers for app builds as only root users can mount filesystems. So a lot of these decisions have all kinds of tradeoffs that should be better known and discussed. [2] Disclaimer: Founder at Flockport and trying to build a simpler alternative with LXC with support for app builds, provisioning, orchestration, service discovery and an app store. [3] [1] https://www.flockport.com/guides/say-yes-to-containers.html https://www.flockport.com/guides/say-yes-to-containers.html [2] https://www.flockport.com/guides/understanding-layers.html https://www.flockport.com/guides/understanding-layers.html [3] https://www.flockport.com/guides/cluster-services https://www.flockport.com/guides/cluster-services
  • vikrantrathore 8y ago
    @tobbyb Thanks for all the work on flockport your tutorials on LXC helped me to have initial production systems ready. I applied many of those concepts specially for networking using flockport tutorial. Although today my system with LXD is much easier and getting better with every subsequent release of LXD, I owe you gratitude for helping me to get off the ground.
    • tobbyb 8y ago
      @vikrant Glad to hear that! Networking in Linux is incredibly feature rich and the LXC team have done a great job on container networking.