6 ms·

LXD is actually a cool technology. It's not like Docker/k8s in terms of each "node" usually running only one thing. It's more along the lines of a "VPS in a box

LXD is actually a cool technology. It's not like Docker/k8s in terms of each "node" usually running only one thing. It's more along the lines of a "VPS in a box", where you can launch virtual servers using a simple command line interface. I use it to run all of the virtual server machines in my home LAN (mostly virtual desktops) with each machine exposed on the real LAN's DHCP server (so I can move them to a different box if needed), and to launch ephemeral "server boxes" to run some tests or builds or whatever on without polluting my dev environment (think virtualenv, but for anything). I also used it as the base for my virtual system builder scripts ("container" VMs with LXD, "VM" VMs with libvirt since LXD didn't support VMs back then): https://github.com/kstenerud/virtual-builders https://github.com/kstenerud/virtual-builders

jaekash 6y ago
> LXD is actually a cool technology. It is a pretty nifty idea but like all things made by Canonical it is basically digitized garbage. I tried to get LXC running on fedora some months, wondering why there are no official packages and then I very quickly noticed why. LXD uses the old Ubuntu trick of calling sysv style init scripts from systemd unit files without so much as a "|| exit 1" [1][2] - and I don't have time for that bullshit. If you want vastly superior quality try podman with --rootfs or virt-install. At least those are not made by people with a disdain for error checking. [1]: https://github.com/lxc/lxc/blob/master/config/init/systemd/lxc.service.in https://github.com/lxc/lxc/blob/master/config/init/systemd/l... [2]: https://github.com/lxc/lxc/blob/master/config/init/common/lxc-containers.in https://github.com/lxc/lxc/blob/master/config/init/common/lx...
- kstenerud 6y ago
  Thanks! That's good to know because I've been having troubles with Ubuntu 20 and have been considering switching back to redhat. Then the only remaining part would be figuring out how to do zfs on root in redhat. I do it with a chroot bash script for Ubuntu server atm, but hopefully there's a debootstrap kind of thing on redhat that I can use as a base.
  - josteink 6y ago
    > I do it with a chroot bash script for Ubuntu server atm I’ve been using ZFS root on Ubuntu without any such nonsense for years, but I had to debootstrap it manually. See guides here: https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubuntu/index.html#root-on-zfs https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubu... With Ubuntu 20.04 the standard installer should support setting up a ZFS root without all that manual work, but I haven’t tested it myself.
    - kstenerud 6y ago
      Yes, that's what I meant. The script does the partitioning, zfs init, chroot, debootstrap, and apt install. You just run it from a live cd. https://github.com/kstenerud/ubuntu-server-zfs https://github.com/kstenerud/ubuntu-server-zfs
  - jaekash 6y ago
    > I do it with a chroot bash script for Ubuntu server atm, but hopefully there's a debootstrap kind of thing on redhat that I can use as a base. There are ways to do chroot installs on RHEL but sadly not well documented. Some resources on this: - https://glacion.com/2019/06/16/Fedora.html https://glacion.com/2019/06/16/Fedora.html - https://github.com/glacion/fedora-chroot-installation https://github.com/glacion/fedora-chroot-installation - https://web.archive.org/web/20150912004438/https://wiki.centos.org/HowTos/ManualInstall https://web.archive.org/web/20150912004438/https://wiki.cent... Most of that will work for RHEL with some minor modifications. Pity it is not better documented.
    - Conan_Kudo 6y ago
      YUM/DNF has bootstrap-in-chroot functionality built-in. Just set up your disk layout the way you want on a disk device or a loopback disk backed by an image, and do something like: $ sudo dnf --installroot=/path/to/mounted/target group install @core @hardware-support And you're good to go. You can see the list of groups available to include by using "dnf group list --ids".
- unixhero 6y ago
  Also superior Proxmox
  - znpy 6y ago
    Yup. I discovered LXC container with proxmox, now i have many of them running in my network.
    - thejosh 6y ago
      Yeah, been playing with proxmox on my homelab, put it on an AMD Ryzen 5 3600 (6 cores, 12 threads), with 32gb of RAM and it's been running great with containers for when I want to fool around. Fantastic piece of software that has matured well.
  - nix23 6y ago
    Proxmox is an absolute great product, in production at 7 clients since over 8 years, some with ZFS some with Ceph, some with lxd some kvm, it's rock-solid and the support is great!
- znpy 6y ago
  In fairness, distros seems to make this kind of "joke" one another. About six months ago I tried running SystemTap on Ubuntu...
  - jaekash 6y ago
    Calling sysv style init scripts with no error checking from systemd unit files is a joke on Canonical, not any other distro.
- rlpb 6y ago
  You're confusing LXC with LXD. LXC is legacy and is mostly left as-is for existing users who are happy [and don't want the LXD paradigm shift]. Think of it as 1.0. You're complaining about stuff that is effectively a maintenance branch. At least your links point to LXC, not LXD. LXD is the new thing. Think of it as 2.0 and onwards.
  - CameronNemo 6y ago
    I'm not sure if that is a good description. LXC is a library and set of command line utilities to run containers. LXD is a clustered service that makes use of the LXC library.
    - rlpb 6y ago
      I admit it is confusing. LXC typically means the lxc-* commands like lxc-start and so on. That's the daemon-less "old way". The links the grandparent provided was to this, and their complaint is about that stuff that is in maintenance mode and not actively developed any more. LXD is a complete rewrite. There is now a server daemon called "lxd", and the client for that is "lxc".
- niemeyer 6y ago
  > It is a pretty nifty idea but like all things made by Canonical it is basically digitized garbage. > At least those are not made by people with a disdain for error checking. Every single time there's a topic about Ubuntu or Canonical you seem to go straight into offensive mode. I can tell you use Fedora, but that's not typical of Fedora users and developers and no excuse. I'm an open source developer and believer, but this kind of behavior slowly burns my soul, even more when it seems accepted. I've worked on enough projects in my life that it's absolutely certain that you use my code regularly. It's inside Go, Python, APT, and RPM by the way (hello Jeff Johnson, wherever you are), and in key libraries that you surely depend on as well. And I never heard you complaining about any of that with such anger here. I'm also Canonical's CTO, and I was one of the key designers and developers that started snaps, and juju, and other key projects from Canonical. I usually hear such blind hate in silence, but sometimes it's just too much. I don't understand why do we do that to ourselves, as a Linux community. Why is it okay to openly offend unknown people that we almost certainly depend upon? What is it that we came here to do, again?
cpsaltis 6y ago
LXD is indeed nice, but it still isn't very widely adopted. I believe this comes down to 3 main issues: 1) It's strongly connected to Canonical and Ubuntu. This is mostly a matter of perception and it is an actual community project. However, I can understand people not feeling comfortable with "snap install lxd". 2) It sits somewhere in between k8s and docker engine. Over time, it will probably get more k8s-like features but still it is a weird position to be in. 3) It lacks a reach ecosystem of tools supporting it and a web UI. This makes it hard for newcomers to adopt. We're working on a web UI ourselves as part of our open source cloud management platform (https://github.com/mistio/mist-ce https://github.com/mistio/mist-ce) and love to hear your thoughts.
- davb 6y ago
  Indeed, we have major issues with snap (doesn't work with $HOME on nfs, and auto updates with very few controls are a terrible idea on servers) so avoid anything dependent on it. Otherwise, I really like the concept.
  - kwk1 6y ago
    It's unfortunate that the effort to get LXD packaged natively in Debian [1] seem to have stalled as this is definitely one of the last remaining drawbacks for me. For example, when LXD was still available as a .deb from upstream, it was possible to run LXD inside a container and do nesting, but with the snap, that isn't possible anymore. (However it is now with the --vm flag, though that's really a different mechanism.) As I understand it the big remaining difficulty is dqlite. [1] https://wiki.debian.org/LXD https://wiki.debian.org/LXD
    - Foxboron 6y ago
      I did write a bit about my experience packaging LXD for Arch Linux a while back. I got a ping from a Debian dev telling me the Debian dev working on LXD enjoyed the article, and has been trying to pick up on the work again. The linked wiki page is the outdated one. https://linderud.dev/blog/packaging-lxd-for-arch-linux/ https://linderud.dev/blog/packaging-lxd-for-arch-linux/

jaekash 6y ago

> LXD is actually a cool technology. It is a pretty nifty idea but like all things made by Canonical it is basically digitized garbage. I tried to get LXC running on fedora some months, wondering why there are no official packages and then I very quickly noticed why. LXD uses the old Ubuntu trick of calling sysv style init scripts from systemd unit files without so much as a "|| exit 1" [1][2] - and I don't have time for that bullshit. If you want vastly superior quality try podman with --rootfs or virt-install. At least those are not made by people with a disdain for error checking. [1]: https://github.com/lxc/lxc/blob/master/config/init/systemd/lxc.service.in https://github.com/lxc/lxc/blob/master/config/init/systemd/l... [2]: https://github.com/lxc/lxc/blob/master/config/init/common/lxc-containers.in https://github.com/lxc/lxc/blob/master/config/init/common/lx...

kstenerud 6y ago
Thanks! That's good to know because I've been having troubles with Ubuntu 20 and have been considering switching back to redhat. Then the only remaining part would be figuring out how to do zfs on root in redhat. I do it with a chroot bash script for Ubuntu server atm, but hopefully there's a debootstrap kind of thing on redhat that I can use as a base.
- josteink 6y ago
  > I do it with a chroot bash script for Ubuntu server atm I’ve been using ZFS root on Ubuntu without any such nonsense for years, but I had to debootstrap it manually. See guides here: https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubuntu/index.html#root-on-zfs https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubu... With Ubuntu 20.04 the standard installer should support setting up a ZFS root without all that manual work, but I haven’t tested it myself.
  - kstenerud 6y ago
    Yes, that's what I meant. The script does the partitioning, zfs init, chroot, debootstrap, and apt install. You just run it from a live cd. https://github.com/kstenerud/ubuntu-server-zfs https://github.com/kstenerud/ubuntu-server-zfs
- jaekash 6y ago
  > I do it with a chroot bash script for Ubuntu server atm, but hopefully there's a debootstrap kind of thing on redhat that I can use as a base. There are ways to do chroot installs on RHEL but sadly not well documented. Some resources on this: - https://glacion.com/2019/06/16/Fedora.html https://glacion.com/2019/06/16/Fedora.html - https://github.com/glacion/fedora-chroot-installation https://github.com/glacion/fedora-chroot-installation - https://web.archive.org/web/20150912004438/https://wiki.centos.org/HowTos/ManualInstall https://web.archive.org/web/20150912004438/https://wiki.cent... Most of that will work for RHEL with some minor modifications. Pity it is not better documented.
  - Conan_Kudo 6y ago
    YUM/DNF has bootstrap-in-chroot functionality built-in. Just set up your disk layout the way you want on a disk device or a loopback disk backed by an image, and do something like: $ sudo dnf --installroot=/path/to/mounted/target group install @core @hardware-support And you're good to go. You can see the list of groups available to include by using "dnf group list --ids".
unixhero 6y ago
Also superior Proxmox
- znpy 6y ago
  Yup. I discovered LXC container with proxmox, now i have many of them running in my network.
  - thejosh 6y ago
    Yeah, been playing with proxmox on my homelab, put it on an AMD Ryzen 5 3600 (6 cores, 12 threads), with 32gb of RAM and it's been running great with containers for when I want to fool around. Fantastic piece of software that has matured well.
- nix23 6y ago
  Proxmox is an absolute great product, in production at 7 clients since over 8 years, some with ZFS some with Ceph, some with lxd some kvm, it's rock-solid and the support is great!
znpy 6y ago
In fairness, distros seems to make this kind of "joke" one another. About six months ago I tried running SystemTap on Ubuntu...
- jaekash 6y ago
  Calling sysv style init scripts with no error checking from systemd unit files is a joke on Canonical, not any other distro.
rlpb 6y ago
You're confusing LXC with LXD. LXC is legacy and is mostly left as-is for existing users who are happy [and don't want the LXD paradigm shift]. Think of it as 1.0. You're complaining about stuff that is effectively a maintenance branch. At least your links point to LXC, not LXD. LXD is the new thing. Think of it as 2.0 and onwards.
- CameronNemo 6y ago
  I'm not sure if that is a good description. LXC is a library and set of command line utilities to run containers. LXD is a clustered service that makes use of the LXC library.
  - rlpb 6y ago
    I admit it is confusing. LXC typically means the lxc-* commands like lxc-start and so on. That's the daemon-less "old way". The links the grandparent provided was to this, and their complaint is about that stuff that is in maintenance mode and not actively developed any more. LXD is a complete rewrite. There is now a server daemon called "lxd", and the client for that is "lxc".
niemeyer 6y ago
> It is a pretty nifty idea but like all things made by Canonical it is basically digitized garbage. > At least those are not made by people with a disdain for error checking. Every single time there's a topic about Ubuntu or Canonical you seem to go straight into offensive mode. I can tell you use Fedora, but that's not typical of Fedora users and developers and no excuse. I'm an open source developer and believer, but this kind of behavior slowly burns my soul, even more when it seems accepted. I've worked on enough projects in my life that it's absolutely certain that you use my code regularly. It's inside Go, Python, APT, and RPM by the way (hello Jeff Johnson, wherever you are), and in key libraries that you surely depend on as well. And I never heard you complaining about any of that with such anger here. I'm also Canonical's CTO, and I was one of the key designers and developers that started snaps, and juju, and other key projects from Canonical. I usually hear such blind hate in silence, but sometimes it's just too much. I don't understand why do we do that to ourselves, as a Linux community. Why is it okay to openly offend unknown people that we almost certainly depend upon? What is it that we came here to do, again?

cpsaltis 6y ago

LXD is indeed nice, but it still isn't very widely adopted. I believe this comes down to 3 main issues: 1) It's strongly connected to Canonical and Ubuntu. This is mostly a matter of perception and it is an actual community project. However, I can understand people not feeling comfortable with "snap install lxd". 2) It sits somewhere in between k8s and docker engine. Over time, it will probably get more k8s-like features but still it is a weird position to be in. 3) It lacks a reach ecosystem of tools supporting it and a web UI. This makes it hard for newcomers to adopt. We're working on a web UI ourselves as part of our open source cloud management platform (https://github.com/mistio/mist-ce https://github.com/mistio/mist-ce) and love to hear your thoughts.

davb 6y ago
Indeed, we have major issues with snap (doesn't work with $HOME on nfs, and auto updates with very few controls are a terrible idea on servers) so avoid anything dependent on it. Otherwise, I really like the concept.
- kwk1 6y ago
  It's unfortunate that the effort to get LXD packaged natively in Debian [1] seem to have stalled as this is definitely one of the last remaining drawbacks for me. For example, when LXD was still available as a .deb from upstream, it was possible to run LXD inside a container and do nesting, but with the snap, that isn't possible anymore. (However it is now with the --vm flag, though that's really a different mechanism.) As I understand it the big remaining difficulty is dqlite. [1] https://wiki.debian.org/LXD https://wiki.debian.org/LXD
  - Foxboron 6y ago
    I did write a bit about my experience packaging LXD for Arch Linux a while back. I got a ping from a Debian dev telling me the Debian dev working on LXD enjoyed the article, and has been trying to pick up on the work again. The linked wiki page is the outdated one. https://linderud.dev/blog/packaging-lxd-for-arch-linux/ https://linderud.dev/blog/packaging-lxd-for-arch-linux/