8 ms·

RaspberryPi 4 home lab with LXD cluster

125 points by ropyeett 6y ago

nexuist 6y ago
LXD is so weird. It feels like the halfway point between chroot and Docker proper. Can someone more familiar with containers explain what its usecase is meant to be? For reference my only interaction with it was for privilege escalation on HackTheBox: https://www.hackingarticles.in/lxd-privilege-escalation/ https://www.hackingarticles.in/lxd-privilege-escalation/
- martinmunk 6y ago
  I run LXD on some machines at work and overall I love it. Think of it as VM hosting, but very little overhead thanks to the shared kernel. (But can for some reason now also be used to manage KVM VM's) Generally I use it for two use-cases, where I need to do minimal work: * Giving co-workers a container they can use to host small utilities on a static IP to share, instead of them hosting it on their own desktop. Think compiler-explorer ect. * Giving co-workers access to our 64 core ThreadRipper for heavy workloads, sandboxed from one another. But a mandatory "F* Snap" needs to be said.
  - Izkata 6y ago
    > but very little overhead thanks to the shared kernel. (But can for some reason now also be used to manage KVM VM's) Which means you can now use the same interface for, say, a Windows VM alongside your lightweight linux containers. [paraphrased from a reddit post, I was wondering something similar a few weeks ago but haven't done it myself]
  - kingosticks 6y ago
    When does the sandboxing become useful between co-workers? We have a bunch of powerful workstations but we just ssh in and use them. Does this facilitate users installing packages system-wide in their sandbox? Honest question, just trying to work out what we might be missing wit our historic setup.
    - martinmunk 6y ago
      > Does this facilitate users installing packages system-wide in their sandbox? This. They can use it as they please, like a second desktop. I can limit their resources, migrate or archive the containers as needed. One guy was training some ML, one was messing with compiler versions, one with Debian packaging tools ect.
- dragonsh 6y ago
  Not sure if you know docker was built on top of LXC container and later moved to write its own library trying to replicate lxc. In general LXD is more secure than docker in its general configuration as containers in LXD are mapped to userid and use shiftfs [1], nothing like this in Docker yet including kubernetes. Usually docker always had more security vulnerabilities than LXD. So LXD is much better than docker except docker in spite of being an inferior solution became popular with marketing money spend on it due to hype. LXD stayed with people who believe in pragmatic simplicity. Docker is plagued by privilege escalation for a very long time. Check the details in general Docker has more vulnerabilities than LXD. [2] [3] [1] https://lwn.net/Articles/687354/ https://lwn.net/Articles/687354/ [2] https://www.cvedetails.com/vulnerability-list/vendor_id-13534/product_id-28125/Docker-Docker.html https://www.cvedetails.com/vulnerability-list/vendor_id-1353... [3] https://www.cvedetails.com/vulnerability-list/vendor_id-13134/product_id-53787/Linuxcontainers-LXD.html https://www.cvedetails.com/vulnerability-list/vendor_id-1313...
  - cheph 6y ago
    > So LXD is much better than docker except docker in spite of being an inferior solution became popular with marketing money spend on it due to hype. This is kind of like saying a tractor is safer than a car. LXD does not have the same feature set as docker or k8s. If all you want is a more secure docker then podman got you covered. If you want more secure k8s then you will have to wait a bit I guess for https://github.com/rootless-containers/usernetes https://github.com/rootless-containers/usernetes but the cri-o runtime for k8s does have a rootless mode. LXD is not a replacement for docker or for k8s as it offers a different feature set from both of those. Last I wanted what one of the features it offered, which is a persistent whole OS container, I tried to install LXD on fedora, and after trying to get lxc running, failing to do so and seeing it's horror show of a systemd setup while debugging [1], I looked elsewhere and instead settled for rootless podman with --rootfs. [1] https://github.com/lxc/lxc/blob/master/config/init/systemd/lxc.service.in https://github.com/lxc/lxc/blob/master/config/init/systemd/l... does something which ubuntu is very fond off and is the main reason why I stopped using it. They call sysv-style init scripts like https://github.com/lxc/lxc/blob/master/config/init/sysvinit/lxc-containers.in https://github.com/lxc/lxc/blob/master/config/init/sysvinit/... from systemd and in my experience this obsecures errors as somethign ends up failing but via systemd it still looks like it is running.
- panpanna 6y ago
  Think halfway between docker and vagrant instead. I think Mark Shuttleworth explained LXD best: https://www.youtube.com/watch?v=0z3yusiCOCk https://www.youtube.com/watch?v=0z3yusiCOCk
- cheph 6y ago
  To me most Ubuntu things like LXD, Mir, snap feels like it was made by someone who did not understand the existing solutions and could not be bothered to understand it. There may be some points where LXD is "better" (they keep bringing up uid/gid mapping) but it really does not provide the same functionality as OCI containers and does not enable the same workflows as OCI containers. LXD is not an alternative to Docker or K8S, it is something different which offers different features. And if we are just talking docker, and not k8s, then all the security you can ever want can be found in podman which by default operates rootless and daemonless and works on stock standard OCI containers. If we are talking k8s there are already runtimes which support rootless operation like cri-o and there are k8s distros that support rootless operation https://github.com/rootless-containers/usernetes https://github.com/rootless-containers/usernetes - these maybe are not as widely used as they should be and work is ongoing but you will soon see more of them I think.
  - dragonsh 6y ago
    LXC existed before Docker and indeed as I said Docker initially built on top of LXC. So LXD is not an afterthought as you tried to put it, it’s way before Docker and k8s. Docker and k8s became popular given marketing money put on them. Also rootless container has been a feature of LXC since 1.0 in 2013-14, which could not be incorporated in Docker as they tried to re-invent the wheels by writing their own libcontainer which eventually resulted in many vulnerabilities which even impacted k8s even in 2019. Still today unless one use a managed version of k8s or use managed service by major cloud provider the infrastructure will be insecure with k8s given most of the Docker images still not tested as rootless containers. Also for a small team it’s pretty hard to have secure self-hosted k8s infrastructure given sheer complexity and moving parts.
- fomine3 6y ago
  Previously LXC/LXD/Zones/Jail was "normal" container (similar use as virtualization) and Docker was weird technology before Docker is getting popular.
yjftsjthsd-h 6y ago
Does LXD still do the thing where it's only distributed via snap and therefore force automatic updates and restart? It's neat tech (especially supporting containers and VMs), but I just can't bring myself to invest in a platform that forces instability like that.
- LeoPanthera 6y ago
  On Ubuntu maybe? If you don't like snap, don't use Ubuntu. LXC/LXD is not Ubuntu specific.
  - yjftsjthsd-h 6y ago
    That's rather the tail wagging the dog, isn't it? Besides, it kinda rules out most of the options; almost nobody packages it naively, probably because their build/vendoring/packaging process is insane (see https://linderud.dev/blog/packaging-lxd-for-arch-linux/ https://linderud.dev/blog/packaging-lxd-for-arch-linux/).
    - stgraber 6y ago
      There are quite a few Linux distributions with native packages in their main repository, that includes ArchLinux, Alpine, Gentoo and OpenSUSE for those that I'm aware of with active package maintainers. There also are packages for Fedora/CentOS/RHEL through maintained COPR repositories and I'm sure I'm forgetting some distros. Go can be a bit annoying to package in general and LXD was made a bit harder by also having a stack of C libraries for some bits, though with 4.6 we kicked out the need for a custom sqlite and for a coroutine library, so on the C side, outside of the C library, it's down to liblxc, libraft and libdqlite making it a bit easier to package. For distributions that have a policy of splitting every single Go package into their own source packages (as is the case with Debian), our recommendation is to stick to the LTS release of LXD which only gets released every 2 years and where the bugfix releases don't normally alter dependencies. The normal feature releases come out every month and those just aren't a good fit when it may take you more than a month to get any new dependency packaged independently first... Anyway, as an upstream, we are very happy to work with packagers to get native packages in as many distribution as possible. We do maintain the snap package ourselves and certainly do enjoy it as an upstream since it gives us very large distribution and release coverage with a single package. Most common criticism of the snap package can be addressed with one of the mechanisms we outlined here: https://discuss.linuxcontainers.org/t/managing-the-lxd-snap/8178 https://discuss.linuxcontainers.org/t/managing-the-lxd-snap/... That being said, at the end of the day, all we care about is that our users get to run an up to date, secure, LXD. How they get is doesn't really matter to us as an upstream :) And it's maybe interesting to point out that the majority of our userbase these days are on Chromebooks, effectively using the Gentoo ebuild package of LXD!
herpderperator 6y ago
What a coincidence, I just finished setting up my 64GB RAM / 2 TB storage Raspberry Pi 4 PoE cluster tonight (only 1 power cable!): https://www.dropbox.com/s/um5mjt8bucfkrbk/picluster.jpg?dl=0 https://www.dropbox.com/s/um5mjt8bucfkrbk/picluster.jpg?dl=0 I was really impressed that I got line speed Ethernet between the Pis, 112MB/s doing a wget of a 1G file from one to another. Things have improved a lot since the original Raspberry Pi from 2012. I need to figure out how to get the 8th Pi working though, since I'm using one of the PoE ports to power a Mikrotik that I'm using as a wireless bridge onto my existing broadcast domain. Technically I don't need the bridge, since each Pi can connect to my wireless AP over 5GHz 802.11ac just fine, but it feels nicer to funnel upstream data through one point. More importantly, it reduces WiFi collisions since there wouldn't be 8 devices broadcasting simultaneously and interfereing with each other (especially at those close distances) if they all needed to download something from the Internet at the same time... like a Docker image for a Kubernetes deployment :)
- youreanidi0t 6y ago
  You spent at least $720 on the Pis not including case, network and console cables, storage, PSU and PDU harness, switch, (missing) thermal management and a management system. Congrats on not saving any money (or power usage) over buying a Ryzen 5 desktop, which would absolutely murder the Pi4 cluster performance-wise. Better memory bandwidth, AES-NI...for about the same wattage. Seriously, go look at the benchmarks. For compression, a Ryzen 3600 is TWENTY times faster than a Pi4, just to give one example. On top of performance you wouldn't have to deal with hardware consoles, slow disk IO, the network config would be virtual and have a fraction of the latency with much greater bandwidth...and 8x8GB of ram is far less flexible than 1x64GB of ram. You could put the whole thing into sleep/suspend mode when you aren't using it, too. Pi4s are cool but I'm so sick of people using Pi's because they're trendy and will get them attention, instead of them actually making sense for the application.
  - danmur 6y ago
    Burn :P One small thing they're nice for is if you want to play with a cluster (e.g. nomad, kubernetes, whatever) on a budget. Of course it's not going to be great bang for buck just for performance but if you're trying out some distributed computing it's nice to be working to real limitations, network and RAM and particular.
dragonsh 6y ago
Has been using LXC and later LXD in our startup. It’s a pragmatic simple container and now VM management platform and can help companies to build vendor neutral cluster and high availability system leveraging the knowledge of HPC and Linux cluster community. Kubernetes is good for google size companies, for startups it’s an additional overhead and tie their products to vendors specific kubernetes’ distributions given the complexity and moving parts. Try the above setup and than see the simplicity. Also give a try to kubernetes and see which one is easier and better able to help. In my view for installing database and blob storage infrastructure as part of the application VM’s are still better than containers.
- cheph 6y ago
  > vendor neutral In what way is LXD vendor neutral? Are there any vendors except canonical that make significant contributions and investments to it?
  - dragonsh 6y ago
    LXC/LXD both are open source projects not dependent on specific vendor or cloud provider to run. Indeed one can use LXC/LXD combination or use Proxmox, OpenStack, Opennebula cloud management platform all of them suppor LXC/LXD now. All these communities using LXC/LXD contribute to it. Funny enough indeed Debian, RedHat, Suse and Canonical contribute to LXC project as well. If you had issues with LXD try asking your questions on LXD forum and community will support you with your issues. It will not need Canonical’s blessings to support you. It’s unlike if you had issues with anthos k8s from google, you need to be a paid subscriber to get support.
panpanna 6y ago
Speaking of LXD, can we talk about the mess that LXD on WSL2 is? Ubuntu 20.04 on WSL2 doesn't use systemd which means no snapd which means no LXD. This happens due to a series of really bad decisions by canonical and Microsoft. The biggest one being "apt install lxd" attempts to install the snap version but fails silently when snapd is missing.
- cheph 6y ago
  Yeah canonical is not too fond of error checking. I dropped ubuntu because it was calling sysv style init scripts from systemd unit files and not checking for errors (similar to what you can see for lxc here https://github.com/lxc/lxc/blob/master/config/init/systemd/lxc.service.in https://github.com/lxc/lxc/blob/master/config/init/systemd/l... and https://github.com/lxc/lxc/blob/master/config/init/sysvinit/lxc-containers.in https://github.com/lxc/lxc/blob/master/config/init/sysvinit/...). So you would get everything appearing fine but when you dig down you find something is dying but because of their bad engineering choices it is incredibly difficult to figure out why.
jwillmer 6y ago
Pi4 is now officially supporting boot from USB. I bought a cheap SSD from China and can confirm that it works flawlessly. I use this configuration for my home lab: https://www.prusaprinters.org/prints/40852-pi4-case-stand https://www.prusaprinters.org/prints/40852-pi4-case-stand Edit: Test results can be found in the link.
- jve 6y ago
  And the cheap SSD would be... ?
  - jwillmer 6y ago
    I paid 25€ for a 120GB KingSpec Portable SSD. A link to the shop I was buying from is in the link above.
vaccinator 6y ago
I love my pi-4 wireguard server... It beats my asus router openvpn.

125 points by ropyeett 6y ago

LXD is so weird. It feels like the halfway point between chroot and Docker proper. Can someone more familiar with containers explain what its usecase is meant to be? For reference my only interaction with it was for privilege escalation on HackTheBox: https://www.hackingarticles.in/lxd-privilege-escalation/ https://www.hackingarticles.in/lxd-privilege-escalation/

martinmunk 6y ago
I run LXD on some machines at work and overall I love it. Think of it as VM hosting, but very little overhead thanks to the shared kernel. (But can for some reason now also be used to manage KVM VM's) Generally I use it for two use-cases, where I need to do minimal work: * Giving co-workers a container they can use to host small utilities on a static IP to share, instead of them hosting it on their own desktop. Think compiler-explorer ect. * Giving co-workers access to our 64 core ThreadRipper for heavy workloads, sandboxed from one another. But a mandatory "F* Snap" needs to be said.
- Izkata 6y ago
  > but very little overhead thanks to the shared kernel. (But can for some reason now also be used to manage KVM VM's) Which means you can now use the same interface for, say, a Windows VM alongside your lightweight linux containers. [paraphrased from a reddit post, I was wondering something similar a few weeks ago but haven't done it myself]
- kingosticks 6y ago
  When does the sandboxing become useful between co-workers? We have a bunch of powerful workstations but we just ssh in and use them. Does this facilitate users installing packages system-wide in their sandbox? Honest question, just trying to work out what we might be missing wit our historic setup.
  - martinmunk 6y ago
    > Does this facilitate users installing packages system-wide in their sandbox? This. They can use it as they please, like a second desktop. I can limit their resources, migrate or archive the containers as needed. One guy was training some ML, one was messing with compiler versions, one with Debian packaging tools ect.
dragonsh 6y ago
Not sure if you know docker was built on top of LXC container and later moved to write its own library trying to replicate lxc. In general LXD is more secure than docker in its general configuration as containers in LXD are mapped to userid and use shiftfs [1], nothing like this in Docker yet including kubernetes. Usually docker always had more security vulnerabilities than LXD. So LXD is much better than docker except docker in spite of being an inferior solution became popular with marketing money spend on it due to hype. LXD stayed with people who believe in pragmatic simplicity. Docker is plagued by privilege escalation for a very long time. Check the details in general Docker has more vulnerabilities than LXD. [2] [3] [1] https://lwn.net/Articles/687354/ https://lwn.net/Articles/687354/ [2] https://www.cvedetails.com/vulnerability-list/vendor_id-13534/product_id-28125/Docker-Docker.html https://www.cvedetails.com/vulnerability-list/vendor_id-1353... [3] https://www.cvedetails.com/vulnerability-list/vendor_id-13134/product_id-53787/Linuxcontainers-LXD.html https://www.cvedetails.com/vulnerability-list/vendor_id-1313...
- cheph 6y ago
  > So LXD is much better than docker except docker in spite of being an inferior solution became popular with marketing money spend on it due to hype. This is kind of like saying a tractor is safer than a car. LXD does not have the same feature set as docker or k8s. If all you want is a more secure docker then podman got you covered. If you want more secure k8s then you will have to wait a bit I guess for https://github.com/rootless-containers/usernetes https://github.com/rootless-containers/usernetes but the cri-o runtime for k8s does have a rootless mode. LXD is not a replacement for docker or for k8s as it offers a different feature set from both of those. Last I wanted what one of the features it offered, which is a persistent whole OS container, I tried to install LXD on fedora, and after trying to get lxc running, failing to do so and seeing it's horror show of a systemd setup while debugging [1], I looked elsewhere and instead settled for rootless podman with --rootfs. [1] https://github.com/lxc/lxc/blob/master/config/init/systemd/lxc.service.in https://github.com/lxc/lxc/blob/master/config/init/systemd/l... does something which ubuntu is very fond off and is the main reason why I stopped using it. They call sysv-style init scripts like https://github.com/lxc/lxc/blob/master/config/init/sysvinit/lxc-containers.in https://github.com/lxc/lxc/blob/master/config/init/sysvinit/... from systemd and in my experience this obsecures errors as somethign ends up failing but via systemd it still looks like it is running.
panpanna 6y ago
Think halfway between docker and vagrant instead. I think Mark Shuttleworth explained LXD best: https://www.youtube.com/watch?v=0z3yusiCOCk https://www.youtube.com/watch?v=0z3yusiCOCk
cheph 6y ago
To me most Ubuntu things like LXD, Mir, snap feels like it was made by someone who did not understand the existing solutions and could not be bothered to understand it. There may be some points where LXD is "better" (they keep bringing up uid/gid mapping) but it really does not provide the same functionality as OCI containers and does not enable the same workflows as OCI containers. LXD is not an alternative to Docker or K8S, it is something different which offers different features. And if we are just talking docker, and not k8s, then all the security you can ever want can be found in podman which by default operates rootless and daemonless and works on stock standard OCI containers. If we are talking k8s there are already runtimes which support rootless operation like cri-o and there are k8s distros that support rootless operation https://github.com/rootless-containers/usernetes https://github.com/rootless-containers/usernetes - these maybe are not as widely used as they should be and work is ongoing but you will soon see more of them I think.
- dragonsh 6y ago
  LXC existed before Docker and indeed as I said Docker initially built on top of LXC. So LXD is not an afterthought as you tried to put it, it’s way before Docker and k8s. Docker and k8s became popular given marketing money put on them. Also rootless container has been a feature of LXC since 1.0 in 2013-14, which could not be incorporated in Docker as they tried to re-invent the wheels by writing their own libcontainer which eventually resulted in many vulnerabilities which even impacted k8s even in 2019. Still today unless one use a managed version of k8s or use managed service by major cloud provider the infrastructure will be insecure with k8s given most of the Docker images still not tested as rootless containers. Also for a small team it’s pretty hard to have secure self-hosted k8s infrastructure given sheer complexity and moving parts.
fomine3 6y ago
Previously LXC/LXD/Zones/Jail was "normal" container (similar use as virtualization) and Docker was weird technology before Docker is getting popular.

yjftsjthsd-h 6y ago

Does LXD still do the thing where it's only distributed via snap and therefore force automatic updates and restart? It's neat tech (especially supporting containers and VMs), but I just can't bring myself to invest in a platform that forces instability like that.

LeoPanthera 6y ago
On Ubuntu maybe? If you don't like snap, don't use Ubuntu. LXC/LXD is not Ubuntu specific.
- yjftsjthsd-h 6y ago
  That's rather the tail wagging the dog, isn't it? Besides, it kinda rules out most of the options; almost nobody packages it naively, probably because their build/vendoring/packaging process is insane (see https://linderud.dev/blog/packaging-lxd-for-arch-linux/ https://linderud.dev/blog/packaging-lxd-for-arch-linux/).
  - stgraber 6y ago
    There are quite a few Linux distributions with native packages in their main repository, that includes ArchLinux, Alpine, Gentoo and OpenSUSE for those that I'm aware of with active package maintainers. There also are packages for Fedora/CentOS/RHEL through maintained COPR repositories and I'm sure I'm forgetting some distros. Go can be a bit annoying to package in general and LXD was made a bit harder by also having a stack of C libraries for some bits, though with 4.6 we kicked out the need for a custom sqlite and for a coroutine library, so on the C side, outside of the C library, it's down to liblxc, libraft and libdqlite making it a bit easier to package. For distributions that have a policy of splitting every single Go package into their own source packages (as is the case with Debian), our recommendation is to stick to the LTS release of LXD which only gets released every 2 years and where the bugfix releases don't normally alter dependencies. The normal feature releases come out every month and those just aren't a good fit when it may take you more than a month to get any new dependency packaged independently first... Anyway, as an upstream, we are very happy to work with packagers to get native packages in as many distribution as possible. We do maintain the snap package ourselves and certainly do enjoy it as an upstream since it gives us very large distribution and release coverage with a single package. Most common criticism of the snap package can be addressed with one of the mechanisms we outlined here: https://discuss.linuxcontainers.org/t/managing-the-lxd-snap/8178 https://discuss.linuxcontainers.org/t/managing-the-lxd-snap/... That being said, at the end of the day, all we care about is that our users get to run an up to date, secure, LXD. How they get is doesn't really matter to us as an upstream :) And it's maybe interesting to point out that the majority of our userbase these days are on Chromebooks, effectively using the Gentoo ebuild package of LXD!

herpderperator 6y ago

What a coincidence, I just finished setting up my 64GB RAM / 2 TB storage Raspberry Pi 4 PoE cluster tonight (only 1 power cable!): https://www.dropbox.com/s/um5mjt8bucfkrbk/picluster.jpg?dl=0 https://www.dropbox.com/s/um5mjt8bucfkrbk/picluster.jpg?dl=0 I was really impressed that I got line speed Ethernet between the Pis, 112MB/s doing a wget of a 1G file from one to another. Things have improved a lot since the original Raspberry Pi from 2012. I need to figure out how to get the 8th Pi working though, since I'm using one of the PoE ports to power a Mikrotik that I'm using as a wireless bridge onto my existing broadcast domain. Technically I don't need the bridge, since each Pi can connect to my wireless AP over 5GHz 802.11ac just fine, but it feels nicer to funnel upstream data through one point. More importantly, it reduces WiFi collisions since there wouldn't be 8 devices broadcasting simultaneously and interfereing with each other (especially at those close distances) if they all needed to download something from the Internet at the same time... like a Docker image for a Kubernetes deployment :)

youreanidi0t 6y ago
You spent at least $720 on the Pis not including case, network and console cables, storage, PSU and PDU harness, switch, (missing) thermal management and a management system. Congrats on not saving any money (or power usage) over buying a Ryzen 5 desktop, which would absolutely murder the Pi4 cluster performance-wise. Better memory bandwidth, AES-NI...for about the same wattage. Seriously, go look at the benchmarks. For compression, a Ryzen 3600 is TWENTY times faster than a Pi4, just to give one example. On top of performance you wouldn't have to deal with hardware consoles, slow disk IO, the network config would be virtual and have a fraction of the latency with much greater bandwidth...and 8x8GB of ram is far less flexible than 1x64GB of ram. You could put the whole thing into sleep/suspend mode when you aren't using it, too. Pi4s are cool but I'm so sick of people using Pi's because they're trendy and will get them attention, instead of them actually making sense for the application.
- danmur 6y ago
  Burn :P One small thing they're nice for is if you want to play with a cluster (e.g. nomad, kubernetes, whatever) on a budget. Of course it's not going to be great bang for buck just for performance but if you're trying out some distributed computing it's nice to be working to real limitations, network and RAM and particular.

dragonsh 6y ago

Has been using LXC and later LXD in our startup. It’s a pragmatic simple container and now VM management platform and can help companies to build vendor neutral cluster and high availability system leveraging the knowledge of HPC and Linux cluster community. Kubernetes is good for google size companies, for startups it’s an additional overhead and tie their products to vendors specific kubernetes’ distributions given the complexity and moving parts. Try the above setup and than see the simplicity. Also give a try to kubernetes and see which one is easier and better able to help. In my view for installing database and blob storage infrastructure as part of the application VM’s are still better than containers.

cheph 6y ago
> vendor neutral In what way is LXD vendor neutral? Are there any vendors except canonical that make significant contributions and investments to it?
- dragonsh 6y ago
  LXC/LXD both are open source projects not dependent on specific vendor or cloud provider to run. Indeed one can use LXC/LXD combination or use Proxmox, OpenStack, Opennebula cloud management platform all of them suppor LXC/LXD now. All these communities using LXC/LXD contribute to it. Funny enough indeed Debian, RedHat, Suse and Canonical contribute to LXC project as well. If you had issues with LXD try asking your questions on LXD forum and community will support you with your issues. It will not need Canonical’s blessings to support you. It’s unlike if you had issues with anthos k8s from google, you need to be a paid subscriber to get support.

panpanna 6y ago

Speaking of LXD, can we talk about the mess that LXD on WSL2 is? Ubuntu 20.04 on WSL2 doesn't use systemd which means no snapd which means no LXD. This happens due to a series of really bad decisions by canonical and Microsoft. The biggest one being "apt install lxd" attempts to install the snap version but fails silently when snapd is missing.

cheph 6y ago
Yeah canonical is not too fond of error checking. I dropped ubuntu because it was calling sysv style init scripts from systemd unit files and not checking for errors (similar to what you can see for lxc here https://github.com/lxc/lxc/blob/master/config/init/systemd/lxc.service.in https://github.com/lxc/lxc/blob/master/config/init/systemd/l... and https://github.com/lxc/lxc/blob/master/config/init/sysvinit/lxc-containers.in https://github.com/lxc/lxc/blob/master/config/init/sysvinit/...). So you would get everything appearing fine but when you dig down you find something is dying but because of their bad engineering choices it is incredibly difficult to figure out why.

jwillmer 6y ago

Pi4 is now officially supporting boot from USB. I bought a cheap SSD from China and can confirm that it works flawlessly. I use this configuration for my home lab: https://www.prusaprinters.org/prints/40852-pi4-case-stand https://www.prusaprinters.org/prints/40852-pi4-case-stand Edit: Test results can be found in the link.

jve 6y ago
And the cheap SSD would be... ?
- jwillmer 6y ago
  I paid 25€ for a 120GB KingSpec Portable SSD. A link to the shop I was buying from is in the link above.

vaccinator 6y ago

I love my pi-4 wireguard server... It beats my asus router openvpn.