3 ms·

LXC (not LXD) is an utter delight compared to the competition. If you have IPv6 it works without any hacks and it’s like having a whole datacenter all controll

LXC (not LXD) is an utter delight compared to the competition. If you have IPv6 it works without any hacks and it’s like having a whole datacenter all controllable at the command line. Anything that inserts itself into iptables feels like a no no. That’s meant I’ve not really put much effort into LXD or Docker beyond discovering they are kind of heavy. The latter had poor IPv6 support the last time I tried to use it (9 months ago.). It’s there, but it felt like a second class citizen. LXD just feels like Ubuntu to LXC’s Debian, so I also didn’t play with it beyond the initial few hours. LXC itself is a joy. I run Alpine, Debian, and Ubuntu depending on my needs. Everything is disposable, with actual data either in github or a filer. I don’t even bother changing the hostname on the VPSs I provision your use it. Boot, install a firewall, create containers, and forget the original host OS even exists. I’d really recommend getting to grips with the low lever (non-LXD) LXC stuff, especially when you are one “vagrant up” away from having LXC tools on your non-Linux OS!

dahfizz 6y ago
I totally agree. People have it stuck in their heads that container == cgroup, and they don't realize how much cruft e.g. docker layers on top. Anything that requires you to do networking on your CPU absolutely kills performance.
inetknght 6y ago
> Anything that inserts itself into iptables feels like a no no. I have a setup which works great with VirtualBox. I have pfSense installed in a guest VM and the host machine routes through it -- without that guest VM running the host machine can't connect anywhere. It's really handy to have a consistent firewall interface despite every host OS having a different idea of what a firewall should do or look like. Docker works with the VirtualBox setup too. I tried to do the same thing with libvirt for four weekends and eventually gave up. I couldn't get libvirt to play nice with iptables at all. LXC is for containers, of course, instead of VMs. But since Docker works with the VM-guest-as-a-router setup, perhaps I'll really give LXC a try too.
Arkanosis 6y ago
LXC has been a life changer for me. Making stuff compatible with multiple distros, making sure the dependency list is complete and experimenting something very quickly without taking the risk to compromise my main system are solved problems, now. Just install the distro with the most recent kernel on the host (you do already if you use arch) and spawn your Debians, Ubuntus, CentOSes in seconds…
bitwize 6y ago
If you're running a modern Linux system, there's always systemd-nspawn. (Not running systemd? Not modern enough. Get with the times, boomer.)
cyphar 6y ago
> LXD just feels like Ubuntu to LXC’s Debian. A bit of a funny thing to say, given that effectively the same group of developers work on LXD and LXC -- and most are employed by Canonical. LXD was never meant to replace LXC, it solves its own set of problems. I personally like using LXD because it reduces the need for me to write my own scripts to do trivial container management (and it can manage images and containers on a ZFS pool by itself), but if you're more comfortable with LXC then you do you. But I disagree that LXC is significantly more "low level" than LXD -- it just requires more manual work and the configuration format is more transparent about the container setup, but you ultimately have the same capabilities in LXD.
- gorgoiler 6y ago
  In your first paragraph you perfectly describe the Ubuntu/Debian relationship! LXD certainly looks great from the documentation. I would use it if I had dynamic container requirements, or had a lot more containers to manage. One odd decision is that, I think, the lxc command line tool — a significant improvement over the lxc-<thing> suite of commanda that come by default with LXC — only ships with lxd. That has a smell about it that’s a bit funny.
  - cyphar 6y ago
    Well, that's because the "lxc" command-line tool manages LXD -- you can't use the "lxc" commands without using LXD. Arguably it should've been called "lxd-client" or "lxdc" but they probably just felt it was too wordy -- most admins that used LXC and wanted to switch to LXD felt more comfortable typing "lxc <command>". And those who used LXC had no need for a top-level "lxc" command because they used the individual programs directly. Again, since the people working on the projects are the same there's nothing wrong with borrowing the name from the other project. :D

dahfizz 6y ago

I totally agree. People have it stuck in their heads that container == cgroup, and they don't realize how much cruft e.g. docker layers on top. Anything that requires you to do networking on your CPU absolutely kills performance.

inetknght 6y ago

> Anything that inserts itself into iptables feels like a no no. I have a setup which works great with VirtualBox. I have pfSense installed in a guest VM and the host machine routes through it -- without that guest VM running the host machine can't connect anywhere. It's really handy to have a consistent firewall interface despite every host OS having a different idea of what a firewall should do or look like. Docker works with the VirtualBox setup too. I tried to do the same thing with libvirt for four weekends and eventually gave up. I couldn't get libvirt to play nice with iptables at all. LXC is for containers, of course, instead of VMs. But since Docker works with the VM-guest-as-a-router setup, perhaps I'll really give LXC a try too.

Arkanosis 6y ago

LXC has been a life changer for me. Making stuff compatible with multiple distros, making sure the dependency list is complete and experimenting something very quickly without taking the risk to compromise my main system are solved problems, now. Just install the distro with the most recent kernel on the host (you do already if you use arch) and spawn your Debians, Ubuntus, CentOSes in seconds…

bitwize 6y ago

If you're running a modern Linux system, there's always systemd-nspawn. (Not running systemd? Not modern enough. Get with the times, boomer.)

cyphar 6y ago

> LXD just feels like Ubuntu to LXC’s Debian. A bit of a funny thing to say, given that effectively the same group of developers work on LXD and LXC -- and most are employed by Canonical. LXD was never meant to replace LXC, it solves its own set of problems. I personally like using LXD because it reduces the need for me to write my own scripts to do trivial container management (and it can manage images and containers on a ZFS pool by itself), but if you're more comfortable with LXC then you do you. But I disagree that LXC is significantly more "low level" than LXD -- it just requires more manual work and the configuration format is more transparent about the container setup, but you ultimately have the same capabilities in LXD.

gorgoiler 6y ago
In your first paragraph you perfectly describe the Ubuntu/Debian relationship! LXD certainly looks great from the documentation. I would use it if I had dynamic container requirements, or had a lot more containers to manage. One odd decision is that, I think, the lxc command line tool — a significant improvement over the lxc-<thing> suite of commanda that come by default with LXC — only ships with lxd. That has a smell about it that’s a bit funny.
- cyphar 6y ago
  Well, that's because the "lxc" command-line tool manages LXD -- you can't use the "lxc" commands without using LXD. Arguably it should've been called "lxd-client" or "lxdc" but they probably just felt it was too wordy -- most admins that used LXC and wanted to switch to LXD felt more comfortable typing "lxc <command>". And those who used LXC had no need for a top-level "lxc" command because they used the individual programs directly. Again, since the people working on the projects are the same there's nothing wrong with borrowing the name from the other project. :D