6 ms·

Listen people, Firecracker is NOT A HYPERVISOR. A hypervisor runs right on the hardware. KVM is a hypervisor. Firecracker is a process that controls KVM. If

by gwd 3y ago

Listen people, Firecracker is NOT A HYPERVISOR. A hypervisor runs right on the hardware. KVM is a hypervisor. Firecracker is a process that controls KVM. If you want to call firecracker (and QEMU, when used in conjunction with KVM) a VMM ("virtual machine monitor") I won't complain. But please please please, we need a word for what KVM and Xen are, and "hypervisor" is the best fit. Stop using that word for a user-level process like Firecracker.

jstarks 3y ago
Keep fighting the good fight, friend. Although I’ll note that the line between a VMM and hypervisor are not always clear. E.g., KVM includes some things that other hypervisors delegate to the VMM (such as instruction completion). And macOS’s hypervisor.framework is almost a pass through to the CPU’s raw capabilities.
MuffinFlavored 3y ago
> virtual machine monitor Is it good to think of libvirt as a virtual machine mointor, or is that more "virtual machine management"?
- zbrozek 3y ago
  I'd love to get a clear explanation of what libvirt actually does. As far as I can tell it's a qemu argument assembler and launcher. For my own use-case, I just launch qemu from systemd unit files: https://wiki.archlinux.org/title/QEMU#With_systemd_service https://wiki.archlinux.org/title/QEMU#With_systemd_service
  - MuffinFlavored 3y ago
    > As far as I can tell it's a qemu argument assembler Raises a few questions to me: Can you use KVM/do KVM stuff without QEMU? Can you do libvirt stuff without QEMU? Hoping the answers to both aren't useless/"technically, but why would you want to?"
    - simcop2387 3y ago
      > Can you use KVM/do KVM stuff without QEMU? Yes there's a few things out there like Firecracker that use KVM without using QEMU. I'm not completely aware of all of them but they do exist > Can you do libvirt stuff without QEMU? Yes it can also manager LXC containers and a few other types like Xen and Bhyve and Virtuozzo, like QEMU without KVM. The without KVM part is important to letting you run VMs that are emulating other architectures than the native one. For a good bit of this, it is "why would you want to" but there are definitely real cases where you'd want to be able to do this. Like the LXC or Virtuozzo support means that you can run lighter weight containers (same underlying tech as Docker essentially) through the same orchestration/management that you use for virtual machines. And the Bhyve support lets you do the same thing for running things on top of FreeBSD (though I've never used it this way) so that a heterogeneous mix of hosts is managed through the same interfaces.
    - jononor 3y ago
      libvirt supports other hypervisors than QEMU, such as VirtualBox, Xen and LXC.
      - MuffinFlavored 3y ago
        I know ChatGPT responses get downvoted here but I fed it this to help me better understand this: > is KVM a hypervisor? is it type 1 or type 2? is QEMU a hypervisor, is it type 1 or type 2? if QEMU is using KVM, is QEMU then not a hypervisor in that use case? Yes, KVM (Kernel-Based Virtual Machine) is indeed a hypervisor. It's a type 1 hypervisor, also known as a "bare metal" hypervisor. This is because KVM directly runs on the host's hardware to control the hardware and to manage guest operating systems. The fact that it's a Linux kernel module that allows the Linux kernel to function as a hypervisor makes it very efficient. QEMU (Quick Emulator) is a bit more complex. By itself, it is technically a type 2 or "hosted" hypervisor, meaning it runs within a conventional operating system environment. QEMU is a generic, open-source machine emulator and virtualizer that can emulate a variety of hardware types and host a range of guest operating systems. However, when QEMU is used with KVM, the picture changes somewhat. In this case, KVM provides the hardware virtualization where it allows the host machine to call CPU instructions of full virtualization. Then, QEMU emulates the hardware resources and provides the user interface for the VM, thus allowing for better performance and usability. It's this combination of KVM's hardware acceleration capabilities and QEMU's emulation capabilities that makes them often used together. In this case, QEMU is not acting purely as a hypervisor; it's providing hardware emulation and user interface for the VMs, while KVM is the part providing the hypervisor functionality. However, we often refer to the combination of "QEMU/KVM" as a unit when talking about this mode of operation.
  - bonzini 3y ago
    The main important point is that Libvirt takes care of privilege separation. It makes sure that if your VM and/or QEMU are broken out of, there are extra layers to prevent getting access to the whole physical machine. For example it runs QEMU as a very limited user and, if you're using SELinux, the QEMU process can hardly read any file other than the vm image file. By contrast the method in the arch wiki runs QEMU as root. QEMU is exposed to all sort of untrusted input, so you really don't want it to run as root. Libvirt also handles cross machine operations such as live migration, and makes it easier to query a bunch of things from QEMU. For more info see https://www.redhat.com/en/blog/all-you-need-know-about-kvm-userspace https://www.redhat.com/en/blog/all-you-need-know-about-kvm-u...
    - zbrozek 3y ago
      Thank you, this is the best explanation of the utility that I've seen. I appreciate it.
- adql 3y ago
  It's a lot of glue to present a consistent interface but it also does the management part. "API to virtualization system" would probably be closest approximation but it also does some more advanced stuff like coordinating cross-host VM migration
arun-mani-j 3y ago
I think you could help me answer the question that has been in my mind for a month :) Is there any article that tells the difference and relationship between KVM, QEMU, libvirt, virt-manager, Xen, Proxmox etc. with their typical use cases?
- reactordev 3y ago
  KVM is kernel-based virtual machine, with libvirt being its API abstraction over all of it. QEMU is a virtual machine host that leverages kvm or software virtualization to spin up machines on the host. virt-manager does the same. Xen is another virtual machine host, like KVM. Proxmox is a virtual machine manager (like QEMU, virt-manager) but is web based. Libvirt will provide abstraction for kvm,qemu,xen Use cases: proxmox web interface exposed on your local network on a KVM Linux box that uses QEMU to manage VM’s. Proxmox will allow you to do that from the web. QEMU is great for single or small fleet of machines but should be automated for any heavy lifting. Proxmox will do that.
  - bonzini 3y ago
    This is almost entirely wrong especially as far as QEMU, Libvirt and virt-manager are concerned. QEMU is a low level process that represents the virtual machine. It has no equivalent in Xen. Using QEMU directly is not a good idea unless your needs for VM configurations change all the time and you hardly reuse VMs. Libvirt is at a higher level than QEMU. It manages the QEMU processes and gives them access to system resources (image files, network interfaces, pass-through PCI devices). It also makes it easy to manage the configuration of your virtual machines and the resources they use. Higher still is virt-manager, which is a GUI interface for libvirt. Proxmox sits at roughly the same level as virt-manager.
    - gabereiser 3y ago
      How? KVM and Xen are kernel level. QEMU uses KVM but also has a software virtualization capability. Libvirt is an API abstraction over it all. virt-manager is a gui app to manage libvirt machines. Proxmox as well. Proxmox VE talks to VMHost via libvirt.
foundry27 3y ago
Nitpick: it’s not accurate to say that a hypervisor, by definition, runs right on the hardware. Xen (as a type-1 hypervisor) has this property; KVM (as a type-2 hypervisor) does not. It’s important to remember that the single core responsibility of a hypervisor is to divide hardware resources and time between VMs, and this decision-making doesn’t require bare-metal. For those unfamiliar, the informal distinction between type-1 and type-2 is that type-1 hypervisors are in direct control of the allocation of all resources of the physical computer, while type-2 hypervisors operate as some combination of being “part of” / “running on” a host operating system, which owns and allocates the resources. KVM (for example) gives privileged directions to the Linux kernel and its virtualization kernel module for how to manage VMs, and the kernel then schedules and allocates the appropriate system resources. Yes, the type-2 hypervisor needs kernel-mode primitives for managing VMs, and the kernel runs right on the hardware, but those primitives aren’t making management decisions for the division of hardware resources and time between VMs. The type-2 hypervisor is making those decisions, and the hypervisor is scheduled by the OS like any other user-mode process.
- veber-alex 3y ago
  KVM is a type-1 hypervisor [1] [1]: https://www.redhat.com/en/topics/virtualization/what-is-KVM https://www.redhat.com/en/topics/virtualization/what-is-KVM
  - monocasa 3y ago
    There's arguments in both directions for something like kvm. Wiki states it pretty well: > The distinction between these two types is not always clear. For instance, KVM and bhyve are kernel modules[6] that effectively convert the host operating system to a type-1 hypervisor.[7] At the same time, since Linux distributions and FreeBSD are still general-purpose operating systems, with applications competing with each other for VM resources, KVM and bhyve can also be categorized as type-2 hypervisors.[8] https://en.wikipedia.org/wiki/Hypervisor#Classification https://en.wikipedia.org/wiki/Hypervisor#Classification
  - compscigirl 3y ago
    Not really, calling KVM a type-1 is a misunderstanding of what the “bare-metal” distinction is referring to. The real difference between the two types is whether the hypervisor owns the hardware or not. In the case of a type-1, the hypervisor runs below the kernel and controls access to the hardware, even for the kernel. In type-2, the hypervisor runs on the kernel, which owns the hardware, and must go through the kernel to use hardware resources.
rdtsc 3y ago
I think people just pick the coolest sounding term. Imagine someone is sharing what they are working on, what’s cooler sounding “I am working on a virtual machine monitor” or “I am working on a hypervisor”. Hypervisor just sounds futuristic and awesome. It’s like with “isomorphic” code. That just sounds much cooler than “js that runs on the client and the server”.
deleted 3y ago
[deleted]

by gwd 3y ago

jstarks 3y ago

Keep fighting the good fight, friend. Although I’ll note that the line between a VMM and hypervisor are not always clear. E.g., KVM includes some things that other hypervisors delegate to the VMM (such as instruction completion). And macOS’s hypervisor.framework is almost a pass through to the CPU’s raw capabilities.

MuffinFlavored 3y ago

> virtual machine monitor Is it good to think of libvirt as a virtual machine mointor, or is that more "virtual machine management"?

zbrozek 3y ago
I'd love to get a clear explanation of what libvirt actually does. As far as I can tell it's a qemu argument assembler and launcher. For my own use-case, I just launch qemu from systemd unit files: https://wiki.archlinux.org/title/QEMU#With_systemd_service https://wiki.archlinux.org/title/QEMU#With_systemd_service
- MuffinFlavored 3y ago
  > As far as I can tell it's a qemu argument assembler Raises a few questions to me: Can you use KVM/do KVM stuff without QEMU? Can you do libvirt stuff without QEMU? Hoping the answers to both aren't useless/"technically, but why would you want to?"
  - simcop2387 3y ago
    > Can you use KVM/do KVM stuff without QEMU? Yes there's a few things out there like Firecracker that use KVM without using QEMU. I'm not completely aware of all of them but they do exist > Can you do libvirt stuff without QEMU? Yes it can also manager LXC containers and a few other types like Xen and Bhyve and Virtuozzo, like QEMU without KVM. The without KVM part is important to letting you run VMs that are emulating other architectures than the native one. For a good bit of this, it is "why would you want to" but there are definitely real cases where you'd want to be able to do this. Like the LXC or Virtuozzo support means that you can run lighter weight containers (same underlying tech as Docker essentially) through the same orchestration/management that you use for virtual machines. And the Bhyve support lets you do the same thing for running things on top of FreeBSD (though I've never used it this way) so that a heterogeneous mix of hosts is managed through the same interfaces.
  - jononor 3y ago
    libvirt supports other hypervisors than QEMU, such as VirtualBox, Xen and LXC.
    - MuffinFlavored 3y ago
      I know ChatGPT responses get downvoted here but I fed it this to help me better understand this: > is KVM a hypervisor? is it type 1 or type 2? is QEMU a hypervisor, is it type 1 or type 2? if QEMU is using KVM, is QEMU then not a hypervisor in that use case? Yes, KVM (Kernel-Based Virtual Machine) is indeed a hypervisor. It's a type 1 hypervisor, also known as a "bare metal" hypervisor. This is because KVM directly runs on the host's hardware to control the hardware and to manage guest operating systems. The fact that it's a Linux kernel module that allows the Linux kernel to function as a hypervisor makes it very efficient. QEMU (Quick Emulator) is a bit more complex. By itself, it is technically a type 2 or "hosted" hypervisor, meaning it runs within a conventional operating system environment. QEMU is a generic, open-source machine emulator and virtualizer that can emulate a variety of hardware types and host a range of guest operating systems. However, when QEMU is used with KVM, the picture changes somewhat. In this case, KVM provides the hardware virtualization where it allows the host machine to call CPU instructions of full virtualization. Then, QEMU emulates the hardware resources and provides the user interface for the VM, thus allowing for better performance and usability. It's this combination of KVM's hardware acceleration capabilities and QEMU's emulation capabilities that makes them often used together. In this case, QEMU is not acting purely as a hypervisor; it's providing hardware emulation and user interface for the VMs, while KVM is the part providing the hypervisor functionality. However, we often refer to the combination of "QEMU/KVM" as a unit when talking about this mode of operation.
- bonzini 3y ago
  The main important point is that Libvirt takes care of privilege separation. It makes sure that if your VM and/or QEMU are broken out of, there are extra layers to prevent getting access to the whole physical machine. For example it runs QEMU as a very limited user and, if you're using SELinux, the QEMU process can hardly read any file other than the vm image file. By contrast the method in the arch wiki runs QEMU as root. QEMU is exposed to all sort of untrusted input, so you really don't want it to run as root. Libvirt also handles cross machine operations such as live migration, and makes it easier to query a bunch of things from QEMU. For more info see https://www.redhat.com/en/blog/all-you-need-know-about-kvm-userspace https://www.redhat.com/en/blog/all-you-need-know-about-kvm-u...
  - zbrozek 3y ago
    Thank you, this is the best explanation of the utility that I've seen. I appreciate it.
adql 3y ago
It's a lot of glue to present a consistent interface but it also does the management part. "API to virtualization system" would probably be closest approximation but it also does some more advanced stuff like coordinating cross-host VM migration

arun-mani-j 3y ago

I think you could help me answer the question that has been in my mind for a month :) Is there any article that tells the difference and relationship between KVM, QEMU, libvirt, virt-manager, Xen, Proxmox etc. with their typical use cases?

reactordev 3y ago
KVM is kernel-based virtual machine, with libvirt being its API abstraction over all of it. QEMU is a virtual machine host that leverages kvm or software virtualization to spin up machines on the host. virt-manager does the same. Xen is another virtual machine host, like KVM. Proxmox is a virtual machine manager (like QEMU, virt-manager) but is web based. Libvirt will provide abstraction for kvm,qemu,xen Use cases: proxmox web interface exposed on your local network on a KVM Linux box that uses QEMU to manage VM’s. Proxmox will allow you to do that from the web. QEMU is great for single or small fleet of machines but should be automated for any heavy lifting. Proxmox will do that.
- bonzini 3y ago
  This is almost entirely wrong especially as far as QEMU, Libvirt and virt-manager are concerned. QEMU is a low level process that represents the virtual machine. It has no equivalent in Xen. Using QEMU directly is not a good idea unless your needs for VM configurations change all the time and you hardly reuse VMs. Libvirt is at a higher level than QEMU. It manages the QEMU processes and gives them access to system resources (image files, network interfaces, pass-through PCI devices). It also makes it easy to manage the configuration of your virtual machines and the resources they use. Higher still is virt-manager, which is a GUI interface for libvirt. Proxmox sits at roughly the same level as virt-manager.
  - gabereiser 3y ago
    How? KVM and Xen are kernel level. QEMU uses KVM but also has a software virtualization capability. Libvirt is an API abstraction over it all. virt-manager is a gui app to manage libvirt machines. Proxmox as well. Proxmox VE talks to VMHost via libvirt.

foundry27 3y ago

Nitpick: it’s not accurate to say that a hypervisor, by definition, runs right on the hardware. Xen (as a type-1 hypervisor) has this property; KVM (as a type-2 hypervisor) does not. It’s important to remember that the single core responsibility of a hypervisor is to divide hardware resources and time between VMs, and this decision-making doesn’t require bare-metal. For those unfamiliar, the informal distinction between type-1 and type-2 is that type-1 hypervisors are in direct control of the allocation of all resources of the physical computer, while type-2 hypervisors operate as some combination of being “part of” / “running on” a host operating system, which owns and allocates the resources. KVM (for example) gives privileged directions to the Linux kernel and its virtualization kernel module for how to manage VMs, and the kernel then schedules and allocates the appropriate system resources. Yes, the type-2 hypervisor needs kernel-mode primitives for managing VMs, and the kernel runs right on the hardware, but those primitives aren’t making management decisions for the division of hardware resources and time between VMs. The type-2 hypervisor is making those decisions, and the hypervisor is scheduled by the OS like any other user-mode process.

veber-alex 3y ago
KVM is a type-1 hypervisor [1] [1]: https://www.redhat.com/en/topics/virtualization/what-is-KVM https://www.redhat.com/en/topics/virtualization/what-is-KVM
- monocasa 3y ago
  There's arguments in both directions for something like kvm. Wiki states it pretty well: > The distinction between these two types is not always clear. For instance, KVM and bhyve are kernel modules[6] that effectively convert the host operating system to a type-1 hypervisor.[7] At the same time, since Linux distributions and FreeBSD are still general-purpose operating systems, with applications competing with each other for VM resources, KVM and bhyve can also be categorized as type-2 hypervisors.[8] https://en.wikipedia.org/wiki/Hypervisor#Classification https://en.wikipedia.org/wiki/Hypervisor#Classification
- compscigirl 3y ago
  Not really, calling KVM a type-1 is a misunderstanding of what the “bare-metal” distinction is referring to. The real difference between the two types is whether the hypervisor owns the hardware or not. In the case of a type-1, the hypervisor runs below the kernel and controls access to the hardware, even for the kernel. In type-2, the hypervisor runs on the kernel, which owns the hardware, and must go through the kernel to use hardware resources.

rdtsc 3y ago

I think people just pick the coolest sounding term. Imagine someone is sharing what they are working on, what’s cooler sounding “I am working on a virtual machine monitor” or “I am working on a hypervisor”. Hypervisor just sounds futuristic and awesome. It’s like with “isomorphic” code. That just sounds much cooler than “js that runs on the client and the server”.

deleted 3y ago

[deleted]