4 ms·

I was under the impression Xen was being phased out across the industry and that KVM/QEMU is pretty much default now

mcdonje 3y ago
Which industry? There are several products that use xen under the hood, like AWS, Citrix, Qubes, Linode, etc. Why would you install an entire fully fledged OS like Linux to manage a company owned Windows machine that a data analyst uses when you can use a type 1 hypervisor instead?
- dale_glass 3y ago
  Linode uses Xen? That explains things. At one point I managed to convince KVM to work with Linode-style disks where /dev/vda is the actual filesystem, no partition table, but it took a fair amount of fiddling. I'm not sure why people don't do that more often, it's just a lot more comfortable when you can expand a host's LV as needed, and fsck and mount from the outside without having to deal with the embedded partition table.
  - shinies 3y ago
    Linode migrated to KVM as well. Seems everyone is slowly moving off Xen!
  - ilyt 3y ago
    >At one point I managed to convince KVM to work with Linode-style disks where /dev/vda is the actual filesystem, no partition table, but it took a fair amount of fiddling. Just use LVM
    - dale_glass 3y ago
      No, I mean: On standard KVM, if you make a VM and give it /dev/vgname/fedora, inside you get /dev/vda as your main disk, and then you have this: /dev/vda1: /boot/efi /dev/vda2: /boot /dev/vda3: LVM What I want is this: Host side -> VM side /dev/vgname/fedora-boot -> /dev/vda /dev/vgname/fedora-root -> /dev/vdb /dev/vgname/fedora-swap -> /dev/vdc LVM on the VM side is superfluous when you're the owner of everything. This way I don't have to deal with the whole rigamarole of /dev/vgname/fedora having a partition table inside. I can just resize/snapshot/mount/fsck every partition from the host directly. I just find it odd that this seems like an unusual configuration and that pretty much no distro seems to want to work this way.
      - ilyt 3y ago
        Oh, I know what you mean, I just value "it booting on anything" over wasting few hundred megs on /boot|EFI partition. We went thru that phase with Xen and it just made it super annoying to migrate to anything else, because for that to work you need hypervisor-specific assist to boot the kernel instead of generic bootloader. Previous admin also had "genius" idea on putting kernels hypervisor-side which made all kinds of messes, as now hypervisor had to had locally every kernel every machine that might run on it needs for boot... > LVM on the VM side is superfluous when you're the owner of everything. I mean, if you IDGAF how disks fill up, sure. I like to split it so, for example, app filling its data dir does not stop /var/log from logging. I did wrote script that auto-resizes LVs and filesystems (up to a given limits) so I just use that. That on my multi-purpose VPS at least In day job we generally don't do LVM for smaller one purpose VMs but do when there is say a database + significant app data, because app filling disk so DB can't work properly is more annoying to fix than app where upload button just stopped working. Database servers usually get "vdc for database, vda for everything else" because that's nice and easy when it comes to giving it more space.
- cowsandmilk 3y ago
  AWS is moving from Xen to KVM.
- shinies 3y ago
  Both Linode and AWS either migrated off Xen or are in the process of it. Qubes (last I checked) wants to move to a VMM independent design. You can build a very minimal host image based on Alpine or something similar to reduce the surface. I'm not sure how this compares to Xen these days though.
- 0xCMP 3y ago
  I believe AWS' newer instances are KVM based because Firecracker requires KVM support.
ilyt 3y ago
You are correct but some places are too big to migrate out/reengineer
generalizations 3y ago
Why is that? Seemed like Xen was the better design, and KVM just had a lot more pragmatism going for it.
0xCMP 3y ago
I wish I could find a nice explainer on this. I heard this too, but not sure why. If I understood correctly: Xen is a more secure design and has had a lot of work put into it (e.g. why Qubes uses Xen vs KVM), but KVM is: faster, in the kernel, and getting a lot more attention generally so in the future anything it's worse at now could get better because of the investment it is getting. All the above is literally me guessing which is why I'd love someone to actually write a proper explanation as to why "KVM >>> Xen"
plam503711 3y ago
Xen provides a great security design and a protocol to do hypercall that makes sense (unlike kvm+virtio which is DMA all the way, with all the plus in terms of simplicity but the bad on the isolation aspects). If I wanted to caricature the situation: KVM is more simple to work with in terms of dev (you have results fast), but kind of "fuck security". Xen is hard from the dev perspective, because it's a more micro kernel by itself, and you can't cheat to have access to the memory, you have to use grant tables (see https://xcp-ng.org/blog/2022/07/27/grant-table-in-xen/ https://xcp-ng.org/blog/2022/07/27/grant-table-in-xen/ ). So if a part of the industry took a shortcut, doesn't mean Xen isn't still relevant :)

Which industry? There are several products that use xen under the hood, like AWS, Citrix, Qubes, Linode, etc. Why would you install an entire fully fledged OS like Linux to manage a company owned Windows machine that a data analyst uses when you can use a type 1 hypervisor instead?

dale_glass 3y ago
Linode uses Xen? That explains things. At one point I managed to convince KVM to work with Linode-style disks where /dev/vda is the actual filesystem, no partition table, but it took a fair amount of fiddling. I'm not sure why people don't do that more often, it's just a lot more comfortable when you can expand a host's LV as needed, and fsck and mount from the outside without having to deal with the embedded partition table.
- shinies 3y ago
  Linode migrated to KVM as well. Seems everyone is slowly moving off Xen!
- ilyt 3y ago
  >At one point I managed to convince KVM to work with Linode-style disks where /dev/vda is the actual filesystem, no partition table, but it took a fair amount of fiddling. Just use LVM
  - dale_glass 3y ago
    No, I mean: On standard KVM, if you make a VM and give it /dev/vgname/fedora, inside you get /dev/vda as your main disk, and then you have this: /dev/vda1: /boot/efi /dev/vda2: /boot /dev/vda3: LVM What I want is this: Host side -> VM side /dev/vgname/fedora-boot -> /dev/vda /dev/vgname/fedora-root -> /dev/vdb /dev/vgname/fedora-swap -> /dev/vdc LVM on the VM side is superfluous when you're the owner of everything. This way I don't have to deal with the whole rigamarole of /dev/vgname/fedora having a partition table inside. I can just resize/snapshot/mount/fsck every partition from the host directly. I just find it odd that this seems like an unusual configuration and that pretty much no distro seems to want to work this way.
    - ilyt 3y ago
      Oh, I know what you mean, I just value "it booting on anything" over wasting few hundred megs on /boot|EFI partition. We went thru that phase with Xen and it just made it super annoying to migrate to anything else, because for that to work you need hypervisor-specific assist to boot the kernel instead of generic bootloader. Previous admin also had "genius" idea on putting kernels hypervisor-side which made all kinds of messes, as now hypervisor had to had locally every kernel every machine that might run on it needs for boot... > LVM on the VM side is superfluous when you're the owner of everything. I mean, if you IDGAF how disks fill up, sure. I like to split it so, for example, app filling its data dir does not stop /var/log from logging. I did wrote script that auto-resizes LVs and filesystems (up to a given limits) so I just use that. That on my multi-purpose VPS at least In day job we generally don't do LVM for smaller one purpose VMs but do when there is say a database + significant app data, because app filling disk so DB can't work properly is more annoying to fix than app where upload button just stopped working. Database servers usually get "vdc for database, vda for everything else" because that's nice and easy when it comes to giving it more space.
cowsandmilk 3y ago
AWS is moving from Xen to KVM.
shinies 3y ago
Both Linode and AWS either migrated off Xen or are in the process of it. Qubes (last I checked) wants to move to a VMM independent design. You can build a very minimal host image based on Alpine or something similar to reduce the surface. I'm not sure how this compares to Xen these days though.
0xCMP 3y ago
I believe AWS' newer instances are KVM based because Firecracker requires KVM support.

ilyt 3y ago

You are correct but some places are too big to migrate out/reengineer

generalizations 3y ago

Why is that? Seemed like Xen was the better design, and KVM just had a lot more pragmatism going for it.

0xCMP 3y ago

I wish I could find a nice explainer on this. I heard this too, but not sure why. If I understood correctly: Xen is a more secure design and has had a lot of work put into it (e.g. why Qubes uses Xen vs KVM), but KVM is: faster, in the kernel, and getting a lot more attention generally so in the future anything it's worse at now could get better because of the investment it is getting. All the above is literally me guessing which is why I'd love someone to actually write a proper explanation as to why "KVM >>> Xen"

plam503711 3y ago

Xen provides a great security design and a protocol to do hypercall that makes sense (unlike kvm+virtio which is DMA all the way, with all the plus in terms of simplicity but the bad on the isolation aspects). If I wanted to caricature the situation: KVM is more simple to work with in terms of dev (you have results fast), but kind of "fuck security". Xen is hard from the dev perspective, because it's a more micro kernel by itself, and you can't cheat to have access to the memory, you have to use grant tables (see https://xcp-ng.org/blog/2022/07/27/grant-table-in-xen/ https://xcp-ng.org/blog/2022/07/27/grant-table-in-xen/ ). So if a part of the industry took a shortcut, doesn't mean Xen isn't still relevant :)