6 ms·

KVM host in a few lines of code

158 points by stargrave 6y ago

zokier 6y ago
There is also this LWN article which covers much of the same ground: https://lwn.net/Articles/658511/ https://lwn.net/Articles/658511/ I do like here that the author here went beyond minimal hello world and also showed how to boot linux
- JoshTriplett 6y ago
  It's remarkable how much traction that article has gotten over the years; it's also cited by crosvm, rust-vmm, Firecracker, and many other virtualization projects.
  - mjb 6y ago
    Thanks for writing it! I've shared that with a lot of people, because it's a great introduction to stepping through the abstraction of QEMU (or libvirt or whatever) and seeing what KVM can do. There's some good articles that start from VMx on up, which are fun too, but much more involved than KVM. In general, this is a kind of technical content I'd like to see more of: here's a lower-level API, what can I do with it?
    - JoshTriplett 6y ago
      I enjoy such articles as well, especially when the lower-level API is something that most people only use through one piece of complex high-level software. Similarly, it's fun to study containers starting with "unshare" and "clone".
      - teknopaul 6y ago
        Top tip, thanks
  - zaxcellent 6y ago
    Indeed it was that lwn article that I used as a reference to write the very first code in crosvm. Excellently written.
st_goliath 6y ago
> However, no one forces you to use KVM directly. There is libvirt, a nice friendly wrapper for low-level virtualization techniques such as KVM or BHyve. No, that's not what libvirt does. libvirt abstracts handling of tools like Qemu, or LXC. No lowlevel stuff. When people talk about libvirt, they usually mean the libvirt daemon that manages VMs or containers using other programs and accepts XML machine descriptions as input. The actual "libvirt" library is simply a front end library for talking to the daemon and is used internally by tools like virsh (CLI) or virtman (GUI tool). Some ~7 years ago I used the libvirt library for a small in-house tool. It has the possibly shittiest API ever designed. No abstraction whatsoever, just a couple C functions that accept XML strings as input and possibly return XML responses from the daemon.
- asguy 6y ago
  > It has the possibly shittiest API ever designed. This was my experience as well. The only time I get near libvirt anymore is to read source; because sometimes it's the best documentation for a qemu feature.
- rwmj 6y ago
  (Part time libvirt developer here ...) I argued early on for libvirt to model things properly through a C API. Obviously I didn't succeed. I eventually wrote a bunch of "interesting" C macros to make it easier to create and submit XML documents: https://github.com/libguestfs/libguestfs-common/blob/master/utils/libxml2-writer-macros.h https://github.com/libguestfs/libguestfs-common/blob/master/... Example usage: https://github.com/libguestfs/libguestfs/blob/b9065fa7adc93123c53f4827e11dad6b210b7d4b/lib/launch-libvirt.c#L1154 https://github.com/libguestfs/libguestfs/blob/b9065fa7adc931... However these days when just about everything is a service accessed by shuffling poorly documented and type-unsafe JSON documents over the network, maybe libvirt was just ahead of its time ... [Libvirt XML is incredibly well documented, probably in some ways over documented.]
- outworlder 6y ago
  > It has the possibly shittiest API ever designed. Used libvirt for LXC, before "Docker" was the new kid on the block. Once we got it working it was great... but libvirt itself, ugh. There were some rubygems that abstracted some of the pain, that helped. Still the ugliness would poke through.
AshamedCaptain 6y ago
It would be trivial for VirtualBox to use KVM instead of shipping an (out of tree) kernel module that uses the vmx instructions directly. I don't understand what exactly is blocking VirtualBox from doing that. They already use HyperV in Windows instead of abusing vmx directly. They could still keep their qemu fork and everything so that the driver compatibility is still top-notch, but at least they would not require out of tree kernel modules, and VirtualBox would also be concurrently usable with other KVM users.
- reilly3000 6y ago
  Methinks IP may have something to do with that, but I'm not sure how.
  - monocasa 6y ago
    I seem to remember that they've said it's because they think that their kernel driver has higher perf than KVM on latency sensitive I/O.
mato 6y ago
If you're interested in "other things" that can be done with KVM, take a look at the Solo5 "hvt" tender: https://github.com/Solo5/solo5/tree/master/tenders/hvt https://github.com/Solo5/solo5/tree/master/tenders/hvt.

158 points by stargrave 6y ago

There is also this LWN article which covers much of the same ground: https://lwn.net/Articles/658511/ https://lwn.net/Articles/658511/ I do like here that the author here went beyond minimal hello world and also showed how to boot linux

JoshTriplett 6y ago
It's remarkable how much traction that article has gotten over the years; it's also cited by crosvm, rust-vmm, Firecracker, and many other virtualization projects.
- mjb 6y ago
  Thanks for writing it! I've shared that with a lot of people, because it's a great introduction to stepping through the abstraction of QEMU (or libvirt or whatever) and seeing what KVM can do. There's some good articles that start from VMx on up, which are fun too, but much more involved than KVM. In general, this is a kind of technical content I'd like to see more of: here's a lower-level API, what can I do with it?
  - JoshTriplett 6y ago
    I enjoy such articles as well, especially when the lower-level API is something that most people only use through one piece of complex high-level software. Similarly, it's fun to study containers starting with "unshare" and "clone".
    - teknopaul 6y ago
      Top tip, thanks
- zaxcellent 6y ago
  Indeed it was that lwn article that I used as a reference to write the very first code in crosvm. Excellently written.

st_goliath 6y ago

> However, no one forces you to use KVM directly. There is libvirt, a nice friendly wrapper for low-level virtualization techniques such as KVM or BHyve. No, that's not what libvirt does. libvirt abstracts handling of tools like Qemu, or LXC. No lowlevel stuff. When people talk about libvirt, they usually mean the libvirt daemon that manages VMs or containers using other programs and accepts XML machine descriptions as input. The actual "libvirt" library is simply a front end library for talking to the daemon and is used internally by tools like virsh (CLI) or virtman (GUI tool). Some ~7 years ago I used the libvirt library for a small in-house tool. It has the possibly shittiest API ever designed. No abstraction whatsoever, just a couple C functions that accept XML strings as input and possibly return XML responses from the daemon.

asguy 6y ago
> It has the possibly shittiest API ever designed. This was my experience as well. The only time I get near libvirt anymore is to read source; because sometimes it's the best documentation for a qemu feature.
rwmj 6y ago
(Part time libvirt developer here ...) I argued early on for libvirt to model things properly through a C API. Obviously I didn't succeed. I eventually wrote a bunch of "interesting" C macros to make it easier to create and submit XML documents: https://github.com/libguestfs/libguestfs-common/blob/master/utils/libxml2-writer-macros.h https://github.com/libguestfs/libguestfs-common/blob/master/... Example usage: https://github.com/libguestfs/libguestfs/blob/b9065fa7adc93123c53f4827e11dad6b210b7d4b/lib/launch-libvirt.c#L1154 https://github.com/libguestfs/libguestfs/blob/b9065fa7adc931... However these days when just about everything is a service accessed by shuffling poorly documented and type-unsafe JSON documents over the network, maybe libvirt was just ahead of its time ... [Libvirt XML is incredibly well documented, probably in some ways over documented.]
outworlder 6y ago
> It has the possibly shittiest API ever designed. Used libvirt for LXC, before "Docker" was the new kid on the block. Once we got it working it was great... but libvirt itself, ugh. There were some rubygems that abstracted some of the pain, that helped. Still the ugliness would poke through.

AshamedCaptain 6y ago

It would be trivial for VirtualBox to use KVM instead of shipping an (out of tree) kernel module that uses the vmx instructions directly. I don't understand what exactly is blocking VirtualBox from doing that. They already use HyperV in Windows instead of abusing vmx directly. They could still keep their qemu fork and everything so that the driver compatibility is still top-notch, but at least they would not require out of tree kernel modules, and VirtualBox would also be concurrently usable with other KVM users.

reilly3000 6y ago
Methinks IP may have something to do with that, but I'm not sure how.
- monocasa 6y ago
  I seem to remember that they've said it's because they think that their kernel driver has higher perf than KVM on latency sensitive I/O.

mato 6y ago

If you're interested in "other things" that can be done with KVM, take a look at the Solo5 "hvt" tender: https://github.com/Solo5/solo5/tree/master/tenders/hvt https://github.com/Solo5/solo5/tree/master/tenders/hvt.