4 ms·
> Full user namespacing has been in from very early on (day 1?) I'm not sure this is true. User namespaces were added a while after LXC first started developme
by cyphar 8y ago
> Full user namespacing has been in from very early on (day 1?)
I'm not sure this is true. User namespaces were added a while after LXC first started development (from memory), and weren't safe (at all) for several years. There was a lot of churn to make user namespaces work (and both LXC and runc had issues with them).
It should be noted that the user namespacing available in LXC is emphatically the correct way of doing it, and few other runtimes have had similarly correct approach. rkt had a similar idea, but never really went all the way with the extra work. Christian Brauner (one of the LXC maintainers) did quite a bit of work to extend the mapping limit for user namespaces just so that LXC could do even more with them.
runc _could_ be used in a similar way to LXC (after all, fro userspace user namespaces are just three files and a clone flag), but you'd need to do the management separately. And doing it right is quite hard when you get in the weeds -- and I absolutely give props to the LXC folks for getting them right.
> IIRC it has not always had support for proper user namespacing for as long as LXC has.
runc has had user namespace support for a shorter time because the project is younger. User namespaces aren't particularly difficult to use (though a lot of the anciliary work is quite significant -- and LXC does a lot of things that are outside the scope of runc).
Most of the delay for Docker to have user namespaces was purely because of Docker (and I even think runc wasn't a separate project when Docker got user namespace support -- it was still libcontainer back then).
> LXC has so many more features packed in I think it's a disservice to compare it to runc in the first place.
I absolutely agree LXC has many more features (many of them that I'm jealous of), it's just that runc is the most apt comparison to it from the Docker stack.
I disagree it's a disservice, given that I know the LXC folks personally and they feel it's okay to make such comparisons (especially when we're discussing individual runtime issues that both runtimes had to deal with in different ways). I'd never claim that runc does things much better than LXC -- it has lots of really great engineering that I wish more people took advantage of.
There are only two things that runc has over LXC:
* In our design, there are no monitor or long-running processes. The upside of this is that you don't have to worry about us being bad at programming -- your container is your business. The downside is that you likely will need to manage your containers somehow (getting the exit code is the most obvious thing you currently need a monitor process for). And console handling is quite complicated. But it does allow for quite a bit of flexibility in the layers above runc -- and it took quite a while to get this right.
* runc has a fully-unprivileged mode ("rootless containers"). Now, LXC can do most of this, but with runc my main idea was that we needed to have completely unprivileged containers (no admin setup, no setuid helpers, and so on). This has pioneered a few projects (slirp4netns -- a highly performant unprivileged user-space veth bridge) which actually now allow you to do an incredibly large number of things without any privileges at any step in the process. So while LXC could do most of this, it used several setuid binaries which made it not work for me -- and runc's rootless container support has really made a lot of strides that wouldn't have been made without the work I did in runc.
But those are fairly minor gripes. LXC even has OCI image support with templates (using a tool I wrote called "umoci" :P), and there were plans to add OCI runtime configuration support as well.
- hardwaresofton 8y agoFirst of all thanks for the in-depth response and all your work on runc! > I'm not sure this is true. User namespaces were added a while after LXC first started development (from memory), and weren't safe (at all) for several years. There was a lot of churn to make user namespaces work (and both LXC and runc had issues with them). Ah, this is news to me -- thanks for clearing that up. I knew that both projects had worked hard on getting user namespaces working and kernel changes did have to land to make it happen. > runc has had user namespace support for a shorter time because the project is younger. User namespaces aren't particularly difficult to use (though a lot of the anciliary work is quite significant -- and LXC does a lot of things that are outside the scope of runc). > Most of the delay for Docker to have user namespaces was purely because of Docker (and I even think runc wasn't a separate project when Docker got user namespace support -- it was still libcontainer back then). Yup, I was thinking of the switch from LXC -> libcontainer -> CRI-shim|containerd|runc which is the reality now. Basically LXD just stayed on LXC and let that mature and got all the benefits that came along. But I also value what runc achieved because the CRI is a fantastic abstraction and fossilization of what people can/should expect from a container runtime, and runc is the minimal realization of those requirements. > It should be noted that the user namespacing available in LXC is emphatically the correct way of doing it, and few other runtimes have had similarly correct approach. rkt had a similar idea, but never really went all the way with the extra work. Christian Brauner (one of the LXC maintainers) did quite a bit of work to extend the mapping limit for user namespaces just so that LXC could do even more with them. Agreed, I actually use rkt for my first few k8s installs back when CoreOS was brandishing "rktnetes". For anyone interested Christian has an excellent talk[0] on this about the stuff in the kernel that had to land for all this magic to be possible. > I absolutely agree LXC has many more features (many of them that I'm jealous of), it's just that runc is the most apt comparison to it from the Docker stack. > I disagree it's a disservice, given that I know the LXC folks personally and they feel it's okay to make such comparisons (especially when we're discussing individual runtime issues that both runtimes had to deal with in different ways). I'd never claim that runc does things much better than LXC -- it has lots of really great engineering that I wish more people took advantage of. I want to be clear that I didn't mean to disparage runc in anyway as a tool, they just seem (to me) to operate at entirely different layers. runc (to me) is small nimble and unix-y -- lxc is like more like a microkernel. I meant it would be a disservice to both tools to compare them. > There are only two things that runc has over LXC: ... Thanks for the from-the-horses-mouth comparison of both! I had no idea you wrote umoci -- thanks for that as well -- it's a super useful tool. [0]: https://www.youtube.com/watch?v=-PZNF8XDNn8 https://www.youtube.com/watch?v=-PZNF8XDNn8