3 ms·

It's an annoying that you can only make snapshots on a stopped container. With VMs it works in a running VM.

by petre 5y ago

nullwarp 5y ago
Also can't do live migrations or backups and moving storage around is a headache. We've pretty much stopped using LXC containers in Proxmox because of all the little issues.
- code_biologist 5y ago
  Dumb question, but what have you replaced it with? I’ve been working on setting up Proxmox/LXC on a box at home as we speak — but if it’s not worth the headache I’ll stick to VMs.
  - doubled112 5y ago
    I'm not who you asked but I use them for different purposes. There are some quirks here and there. Never anything that won't run, but it could potentially add extra troubleshooting. Things that got me recently, just off the top of my head: - htop sees memory usage incorrectly - Docker tries to use overlay2 on ZFS which fails (I think, I needed to create and mount an ext4 volume for reasons) - Hashicorp Vault needed disable_mlock because I believe LXC blocks the syscall. On the other hand, I like my Samba file server in a container though, because it is much easier to share storage from the host into LXC than a VM. LXC and VMs both have pros and cons.
razerbeans 5y ago
Weird, I just tested on my proxmox instance and I was able to create a snapshot of a running container (PVE 7.1-10)
- LilBytes 5y ago
  Yeah, I haven't got these issues either.
aaronius 5y ago
That highly depends on the underlying storage. If it is something that supports snapshots (ZFS, Ceph, LVM thin) then it should work fine, also backups will be possible without any downtime as they will be read from a temporary snapshot.
- buybackoff 5y ago
  Even with ZFS you still have to wait for RAM to dump, haven't you? And it will freeze at least for the dump write time. Do they have CoW for container memory? But even if they had, the RAM snapshot needs to be written, but without freezing the container. I would appreciate an option when I could ignore everything that was not fsyned, e.g. Postgres use case. In that case the normal ZFS snapshot should be enough.
  - aaronius 5y ago
    RAM and other state can be part of a snapshot for VMs, in which case the VM will continue right where it was. The state of a container is not part of the snapshot (just checked), as it is really hard to capture the state of the container (CPU, network, all kinds of file and socket handles) and restore it because all an LXC container is, is local processes in their separate cgroups. This is also the reason why a live migration is not really possible right now, as all that would need to be cut out from the current host machine and restore in the target machine. This is much easier for VMs as Qemu offers a nice abstraction layer.

It's an annoying that you can only make snapshots on a stopped container. With VMs it works in a running VM.

by petre 5y ago

It's an annoying that you can only make snapshots on a stopped container. With VMs it works in a running VM.

Also can't do live migrations or backups and moving storage around is a headache. We've pretty much stopped using LXC containers in Proxmox because of all the little issues.

code_biologist 5y ago
Dumb question, but what have you replaced it with? I’ve been working on setting up Proxmox/LXC on a box at home as we speak — but if it’s not worth the headache I’ll stick to VMs.
- doubled112 5y ago
  I'm not who you asked but I use them for different purposes. There are some quirks here and there. Never anything that won't run, but it could potentially add extra troubleshooting. Things that got me recently, just off the top of my head: - htop sees memory usage incorrectly - Docker tries to use overlay2 on ZFS which fails (I think, I needed to create and mount an ext4 volume for reasons) - Hashicorp Vault needed disable_mlock because I believe LXC blocks the syscall. On the other hand, I like my Samba file server in a container though, because it is much easier to share storage from the host into LXC than a VM. LXC and VMs both have pros and cons.

razerbeans 5y ago

Weird, I just tested on my proxmox instance and I was able to create a snapshot of a running container (PVE 7.1-10)

LilBytes 5y ago
Yeah, I haven't got these issues either.

aaronius 5y ago

That highly depends on the underlying storage. If it is something that supports snapshots (ZFS, Ceph, LVM thin) then it should work fine, also backups will be possible without any downtime as they will be read from a temporary snapshot.

buybackoff 5y ago
Even with ZFS you still have to wait for RAM to dump, haven't you? And it will freeze at least for the dump write time. Do they have CoW for container memory? But even if they had, the RAM snapshot needs to be written, but without freezing the container. I would appreciate an option when I could ignore everything that was not fsyned, e.g. Postgres use case. In that case the normal ZFS snapshot should be enough.
- aaronius 5y ago
  RAM and other state can be part of a snapshot for VMs, in which case the VM will continue right where it was. The state of a container is not part of the snapshot (just checked), as it is really hard to capture the state of the container (CPU, network, all kinds of file and socket handles) and restore it because all an LXC container is, is local processes in their separate cgroups. This is also the reason why a live migration is not really possible right now, as all that would need to be cut out from the current host machine and restore in the target machine. This is much easier for VMs as Qemu offers a nice abstraction layer.