6 ms·

QEMU 8.0

295 points by neustradamus 4y ago

tryauuum 4y ago
experimental VFIO migration? I guess this will make easier to migrate a VM with GPU workload from host to host without interrupting it
- CoastalCoder 4y ago
  Are there GPUs that allow enough internal state to be dumped / reloaded for a migration like that to work? (I'm asking from pure ignorance.)
  - monocasa 4y ago
    You can do it in the vfio driver by just keeping the bookkeeping information around as resources are created.
  - tryauuum 4y ago
    I'm also ignorant about this topic, but it's certainly accomplishable. There's a (quite old) video with demonstration: https://youtu.be/jjkcPn19fcs https://youtu.be/jjkcPn19fcs
orra 4y ago
> x86: support for Xen guests under KVM with Linux v5.12+ Clearly I haven't been keeping up, because this is a bit of a surprise. Xen under KVM? Those things are polar opposites!
- wmf 4y ago
  Xen is kind of dead so people are migrating to KVM but they don't want to update their images.
  - luckyobserver 4y ago
    Doesn't AWS run on a modified Xen setup?
    - monocasa 4y ago
      They used to, but they run mainly on KVM now with a custom vmm.
      - usr1106 4y ago
        First web hit I get on that topic is https://www.freecodecamp.org/news/aws-just-announced-a-move-from-xen-towards-kvm-so-what-is-kvm/ https://www.freecodecamp.org/news/aws-just-announced-a-move-... (I read only the headline and the date now. From Xen towards KVM starting in 2019. So I guess at AWS scale that's still a relatively new thing.)
        monocasa 4y ago
        They started the transition back in 2017. https://www.theregister.com/2017/11/07/aws_writes_new_kvm_based_hypervisor_to_make_its_cloud_go_faster/ https://www.theregister.com/2017/11/07/aws_writes_new_kvm_ba...
        usr1106 4y ago
        Calling systemd-detect-virt on the 2 first AWS machines of different type I have access to: Result is xen on one of them and amazon on the other.
        monocasa 4y ago
        This qemu support comes after Amazon added support to the kernel to emulate Xen when combined with their custom vmm. A key component of that is the hypervisor lying when it's asked by a guest to describe itself. There's a good chance that 'xen' box is actually KVM and Amazon's proprietary vmm.
- monocasa 4y ago
  Support for Xen guests by supporting the Xen hypercalls. This was added to KVM proper to support Amazon's proprietary vmm running ancient AMIs on newer systems but it seems it's finally getting qemu support too.
  - gwd 4y ago
    I haven't looked at this work in detail, but you need more than just hypercalls to run those ancient guests; those guests are expecting to see xenstore, disk and block backends, and so on. Those would be a lot harder to emulate in the kernel (and probably a bad idea to try anyway).
    - monocasa 4y ago
      I mean, sure, this specific work is in qemu, not the kernel. Some hypercalls are trapped by the kernel, some trap out to a vmm like qemu. Support for pv devices mainly lives in the vmm. But A) even the hypercalls the kernel handles needs to be manually enabled by the vmm, and B) the pv device support ultimately is exposed as in guest memory tables and hypercalls (now partially emulated by qemu).
- davidjfelix 4y ago
  My interpretation of this: You have an x86 machine running Linux v5.12 which is running QEMU using KVM accelerations. Using this setup, you're able to emulate a machine that can run Xen. Since Xen is a type 1 hypervisor, you would then run a Dom0 and perhaps many DomU machines inside it, let's say Linux here for both. Hardware > Linux KVM > QEMU > Xen guest > Linux Dom0 guest, Linux DomU guest
  - monocasa 4y ago
    My understanding is that this work is designed to run DomU images more or less directly. Qemu itself more or less fills the role of Dom0 here.
    - davidjfelix 4y ago
      That makes more sense.
  - sureglymop 4y ago
    I hope so because it might finally allow me to run Qubes OS as a vm to check it out.
    - bonzini 4y ago
      That's nested virtualization and it's completely unrelated.
- rbanffy 4y ago
  I always looked at Xen as something similar to a plain VM image with lots and lots of virtio-like stuff. I'd even assume they'd perform better if they are a thinner layer of virtual machine on top of the host.
vrglvrglvrgl 4y ago
[dead]
kissiel 4y ago
"Since version 3.0.0, QEMU uses a time based version numbering scheme: major incremented by 1 for the first release of the year minor reset to 0 with every major increment, otherwise incremented by 1 for each release from git master micro always 0 for releases from git master, incremented by 1 for each stable branch release" If I'm reading this right, "master" has more significant part than stable... This feels wrong. What am I missing?
- bonzini 4y ago
  Next release from 8.0 branch will be 8.0.1 even if it's released in 2024.
rlyshw 4y ago
Cool. I've actually been playing with QEMU internals a lot recently. Specifically with the multi-process experimental features. Although I can't seem to find any consistency on where the main project is headed. They admit that the documentation pages can be well out-of-date with the upstream implementations, but they seem split-brained even within the code. The main project ships with the multi-process qemu approach, mostly defined in their docs: https://www.qemu.org/docs/master/system/multi-process.html https://www.qemu.org/docs/master/system/multi-process.html https://www.qemu.org/docs/master/devel/multi-process.html https://www.qemu.org/docs/master/devel/multi-process.html But I saw some update buried in a mailing list that development of the multi-process system has been superseded by vfio-user, mostly led by nutanix: https://github.com/nutanix/libvfio-user https://github.com/nutanix/libvfio-user The nutanix repo refers to an oracle-led fork of qemu with the full vfio-user implementation built-in: https://github.com/oracle/qemu https://github.com/oracle/qemu So, they're still separate projects, right? Well, kinda. the mainline project has the vfio-user-server implementation merged in: https://github.com/qemu/qemu/blob/ac5f7bf8e208cd7893dbb1a9520559e569a4677c/hw/remote/vfio-user-obj.c https://github.com/qemu/qemu/blob/ac5f7bf8e208cd7893dbb1a952... But not the client side (vfio-user-pci). So, the feature is half-baked in the mainline project. I don't know if any of the qemu devs browse HN but it would be nice to hear more about the plans for vfio-user.

295 points by neustradamus 4y ago

experimental VFIO migration? I guess this will make easier to migrate a VM with GPU workload from host to host without interrupting it

CoastalCoder 4y ago
Are there GPUs that allow enough internal state to be dumped / reloaded for a migration like that to work? (I'm asking from pure ignorance.)
- monocasa 4y ago
  You can do it in the vfio driver by just keeping the bookkeeping information around as resources are created.
- tryauuum 4y ago
  I'm also ignorant about this topic, but it's certainly accomplishable. There's a (quite old) video with demonstration: https://youtu.be/jjkcPn19fcs https://youtu.be/jjkcPn19fcs

orra 4y ago

> x86: support for Xen guests under KVM with Linux v5.12+ Clearly I haven't been keeping up, because this is a bit of a surprise. Xen under KVM? Those things are polar opposites!

wmf 4y ago
Xen is kind of dead so people are migrating to KVM but they don't want to update their images.
- luckyobserver 4y ago
  Doesn't AWS run on a modified Xen setup?
  - monocasa 4y ago
    They used to, but they run mainly on KVM now with a custom vmm.
    - usr1106 4y ago
      First web hit I get on that topic is https://www.freecodecamp.org/news/aws-just-announced-a-move-from-xen-towards-kvm-so-what-is-kvm/ https://www.freecodecamp.org/news/aws-just-announced-a-move-... (I read only the headline and the date now. From Xen towards KVM starting in 2019. So I guess at AWS scale that's still a relatively new thing.)
      - monocasa 4y ago
        They started the transition back in 2017. https://www.theregister.com/2017/11/07/aws_writes_new_kvm_based_hypervisor_to_make_its_cloud_go_faster/ https://www.theregister.com/2017/11/07/aws_writes_new_kvm_ba...
        usr1106 4y ago
        Calling systemd-detect-virt on the 2 first AWS machines of different type I have access to: Result is xen on one of them and amazon on the other.
        monocasa 4y ago
        This qemu support comes after Amazon added support to the kernel to emulate Xen when combined with their custom vmm. A key component of that is the hypervisor lying when it's asked by a guest to describe itself. There's a good chance that 'xen' box is actually KVM and Amazon's proprietary vmm.
monocasa 4y ago
Support for Xen guests by supporting the Xen hypercalls. This was added to KVM proper to support Amazon's proprietary vmm running ancient AMIs on newer systems but it seems it's finally getting qemu support too.
- gwd 4y ago
  I haven't looked at this work in detail, but you need more than just hypercalls to run those ancient guests; those guests are expecting to see xenstore, disk and block backends, and so on. Those would be a lot harder to emulate in the kernel (and probably a bad idea to try anyway).
  - monocasa 4y ago
    I mean, sure, this specific work is in qemu, not the kernel. Some hypercalls are trapped by the kernel, some trap out to a vmm like qemu. Support for pv devices mainly lives in the vmm. But A) even the hypercalls the kernel handles needs to be manually enabled by the vmm, and B) the pv device support ultimately is exposed as in guest memory tables and hypercalls (now partially emulated by qemu).
davidjfelix 4y ago
My interpretation of this: You have an x86 machine running Linux v5.12 which is running QEMU using KVM accelerations. Using this setup, you're able to emulate a machine that can run Xen. Since Xen is a type 1 hypervisor, you would then run a Dom0 and perhaps many DomU machines inside it, let's say Linux here for both. Hardware > Linux KVM > QEMU > Xen guest > Linux Dom0 guest, Linux DomU guest
- monocasa 4y ago
  My understanding is that this work is designed to run DomU images more or less directly. Qemu itself more or less fills the role of Dom0 here.
  - davidjfelix 4y ago
    That makes more sense.
- sureglymop 4y ago
  I hope so because it might finally allow me to run Qubes OS as a vm to check it out.
  - bonzini 4y ago
    That's nested virtualization and it's completely unrelated.
rbanffy 4y ago
I always looked at Xen as something similar to a plain VM image with lots and lots of virtio-like stuff. I'd even assume they'd perform better if they are a thinner layer of virtual machine on top of the host.

vrglvrglvrgl 4y ago

[dead]

kissiel 4y ago

"Since version 3.0.0, QEMU uses a time based version numbering scheme: major incremented by 1 for the first release of the year minor reset to 0 with every major increment, otherwise incremented by 1 for each release from git master micro always 0 for releases from git master, incremented by 1 for each stable branch release" If I'm reading this right, "master" has more significant part than stable... This feels wrong. What am I missing?

bonzini 4y ago
Next release from 8.0 branch will be 8.0.1 even if it's released in 2024.

rlyshw 4y ago

Cool. I've actually been playing with QEMU internals a lot recently. Specifically with the multi-process experimental features. Although I can't seem to find any consistency on where the main project is headed. They admit that the documentation pages can be well out-of-date with the upstream implementations, but they seem split-brained even within the code. The main project ships with the multi-process qemu approach, mostly defined in their docs: https://www.qemu.org/docs/master/system/multi-process.html https://www.qemu.org/docs/master/system/multi-process.html https://www.qemu.org/docs/master/devel/multi-process.html https://www.qemu.org/docs/master/devel/multi-process.html But I saw some update buried in a mailing list that development of the multi-process system has been superseded by vfio-user, mostly led by nutanix: https://github.com/nutanix/libvfio-user https://github.com/nutanix/libvfio-user The nutanix repo refers to an oracle-led fork of qemu with the full vfio-user implementation built-in: https://github.com/oracle/qemu https://github.com/oracle/qemu So, they're still separate projects, right? Well, kinda. the mainline project has the vfio-user-server implementation merged in: https://github.com/qemu/qemu/blob/ac5f7bf8e208cd7893dbb1a9520559e569a4677c/hw/remote/vfio-user-obj.c https://github.com/qemu/qemu/blob/ac5f7bf8e208cd7893dbb1a952... But not the client side (vfio-user-pci). So, the feature is half-baked in the mainline project. I don't know if any of the qemu devs browse HN but it would be nice to hear more about the plans for vfio-user.