4 ms·

What is the definitive new-comer friendly guide to QEMU? Not just about using it but also understanding its internals (say to add new instructions to a supporte

What is the definitive new-comer friendly guide to QEMU? Not just about using it but also understanding its internals (say to add new instructions to a supported ISA etc)?

sipofwater 2y ago
https://www.qemu.org/documentation/ https://www.qemu.org/documentation/
- checker659 2y ago
  I am looking for some hand-holding with this. The documentation seems more for reference (albeit a bit lack-luster if I may say so).
  - bonzini 2y ago
    If there are any specific topics that you'd like covered, please ask.
cpach 2y ago
What are your goals? Are you interested in emulation (i.e. running a VM that uses another architecture than your physical computer) or in virtualization? (i.e. running a VM that uses the same architecture)
- checker659 2y ago
  Well both. I'm interested in implementing an ISA extension (not sure if you know about CHERI). Also, there are reference implementations for aarch64 and risc-v that I'd like to understand.
- hnthrowaway0328 2y ago
  Just curious, what about simulation? I heard that simulation is more serious than emulation and targets for say pipeline level emulation, but maybe it's just a fancier word?
  - cpach 2y ago
    Beats me. What does simulation mean in this context?
    - hnthrowaway0328 2y ago
      Ah nevermind then, probably just a synonym for emulation.
pm215 2y ago
Unfortuately there is none. QEMU is a large project and doesn't have much formal design or API documentation. On the other hand it's not big enough (compared for instance to the Linux kernel) to have a wider community interested in trying to provide internals documentation for newcomers. Our general advice is "look at the existing code for the bit you're interested in to see how it works". You can sometimes find descriptions of the overall architecture online in third party blog posts and the like, but if they're more than a few years old then be wary that they might be out of date -- they're likely to be right in general principles and wrong in details, because things change. For adding new instructions to an existing ISA: the first couple of sections of https://www.qemu.org/docs/master/devel/index-tcg.html https://www.qemu.org/docs/master/devel/index-tcg.html are relevant here. Depending on the target it might or might not use decodetree (decodetree is much easier to add a new insn to, but some older targets still do by-hand switch-statement based decoding.) Look at how an existing insn that is similar to what you want to do works. Implementing CHERI in particular is going to be pretty awful, because the things it does (like 128-bit pointers) break various assumptions QEMU makes. The University of Cambridge forked QEMU to add CHERI support for MIPS and RISC-V and I think also AArch64: https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/cheri-qemu.html https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/cheri... -- but the changes are pretty invasive and also not likely to be very fast. (The fork looks like it's based on 6.0, so three years old now.) (If anybody is interested in trying to write up some documentation for QEMU's internals (either a general overview/roadmap or something on a particular subsystem), I'd be happy to code-review patches that add something to the "Developer Information" subsection of our manual.)
- cpach 2y ago
  Out of curiosity, how does one reach you? (Saw no info/contact details in your profile.) Also, where does QEMU people hang out online? AFAICT the IRC channel is not very active. (Based on few and random visits, so I could be wrong.)
  - pm215 2y ago
    The primary nexus for QEMU developers is the qemu-devel mailing list. (Very high traffic because it's also used for patchmails.) The irc channel is a bit more variable and tends in particular to be quiet outside UK/Europe working hours, just because most QEMU devs happen to be Europe based. I discourage private emails sent direct to me on QEMU topics (because they should generally be to public lists so other community members can answer them or benefit from the answer), but you can find me on the mailing lists and irc.
candiddevmike 2y ago
Start with libvirt, it provides a full GUI around QEMU operations. Run ps to see the underlying QEMU commands it runs. Inspect the XML files to understand how it builds machines.
- cpach 2y ago
  A coworker came up with a similar idea: We started a VM using Lima, then ran ps to see what args it passed to QEMU. It was enlightening!
- freedomben 2y ago
  I agree but I think you meant "Virt Manager"? afaik libvirt is an API, not even a CLI and definitely not a full GUI. For a CLI, virsh is your guy.
  - khrbtxyz 2y ago
    There's also virt-install, which is part of the Virt Manager package. https://github.com/virt-manager/virt-manager/blob/main/man/virt-install.rst https://github.com/virt-manager/virt-manager/blob/main/man/v... curl --output-dir $HOME/.local/share/libvirt/images/ -LO https://cloud.debian.org/images/cloud/bookworm/20240507-1740/debian-12-genericcloud-arm64-20240507-1740.qcow2 virt-install --import --osinfo debian12 --disk size=20,backing_store=$HOME/.local/share/libvirt/images/debian-12-genericcloud-arm64-20240507-1740.qcow2 --controller type=scsi,model=virtio-scsi --cloud-init
Neywiny 2y ago
I implemented a bit of a STM32 and it was a chore and a half. I've noticed 2 things with the code base: 1. It's C but they really want C++. Qemu wrote their own class system, foreach loops, containers, etc. And because of that, when I tried to use actual cpp, compilation failed due to how many reserved keywords were used in headers and other mess. 2. As noted in other comments, copy paste and modify. It got me where I need to go. But it was a slog. Eventually I had gdb debugging my qemu build, and gdb debugging the program I was running. I could even connect in from the STM32 IDE which was nice
skrtskrt 2y ago
probably just start using Proxmox, as it's a pretty beginner-friendly FOSS hypervisor with extensive docs and forums, and it's largely a wrapper around qemu. Their docs often include equivalent qemu commands for any UI actions. For anything the UI can't do yet, they only give the QEMU command

What is the definitive new-comer friendly guide to QEMU? Not just about using it but also understanding its internals (say to add new instructions to a supported ISA etc)?

sipofwater 2y ago

https://www.qemu.org/documentation/ https://www.qemu.org/documentation/

checker659 2y ago
I am looking for some hand-holding with this. The documentation seems more for reference (albeit a bit lack-luster if I may say so).
- bonzini 2y ago
  If there are any specific topics that you'd like covered, please ask.

cpach 2y ago

What are your goals? Are you interested in emulation (i.e. running a VM that uses another architecture than your physical computer) or in virtualization? (i.e. running a VM that uses the same architecture)

checker659 2y ago
Well both. I'm interested in implementing an ISA extension (not sure if you know about CHERI). Also, there are reference implementations for aarch64 and risc-v that I'd like to understand.
hnthrowaway0328 2y ago
Just curious, what about simulation? I heard that simulation is more serious than emulation and targets for say pipeline level emulation, but maybe it's just a fancier word?
- cpach 2y ago
  Beats me. What does simulation mean in this context?
  - hnthrowaway0328 2y ago
    Ah nevermind then, probably just a synonym for emulation.

pm215 2y ago

Unfortuately there is none. QEMU is a large project and doesn't have much formal design or API documentation. On the other hand it's not big enough (compared for instance to the Linux kernel) to have a wider community interested in trying to provide internals documentation for newcomers. Our general advice is "look at the existing code for the bit you're interested in to see how it works". You can sometimes find descriptions of the overall architecture online in third party blog posts and the like, but if they're more than a few years old then be wary that they might be out of date -- they're likely to be right in general principles and wrong in details, because things change. For adding new instructions to an existing ISA: the first couple of sections of https://www.qemu.org/docs/master/devel/index-tcg.html https://www.qemu.org/docs/master/devel/index-tcg.html are relevant here. Depending on the target it might or might not use decodetree (decodetree is much easier to add a new insn to, but some older targets still do by-hand switch-statement based decoding.) Look at how an existing insn that is similar to what you want to do works. Implementing CHERI in particular is going to be pretty awful, because the things it does (like 128-bit pointers) break various assumptions QEMU makes. The University of Cambridge forked QEMU to add CHERI support for MIPS and RISC-V and I think also AArch64: https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/cheri-qemu.html https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/cheri... -- but the changes are pretty invasive and also not likely to be very fast. (The fork looks like it's based on 6.0, so three years old now.) (If anybody is interested in trying to write up some documentation for QEMU's internals (either a general overview/roadmap or something on a particular subsystem), I'd be happy to code-review patches that add something to the "Developer Information" subsection of our manual.)

cpach 2y ago
Out of curiosity, how does one reach you? (Saw no info/contact details in your profile.) Also, where does QEMU people hang out online? AFAICT the IRC channel is not very active. (Based on few and random visits, so I could be wrong.)
- pm215 2y ago
  The primary nexus for QEMU developers is the qemu-devel mailing list. (Very high traffic because it's also used for patchmails.) The irc channel is a bit more variable and tends in particular to be quiet outside UK/Europe working hours, just because most QEMU devs happen to be Europe based. I discourage private emails sent direct to me on QEMU topics (because they should generally be to public lists so other community members can answer them or benefit from the answer), but you can find me on the mailing lists and irc.

candiddevmike 2y ago

Start with libvirt, it provides a full GUI around QEMU operations. Run ps to see the underlying QEMU commands it runs. Inspect the XML files to understand how it builds machines.

cpach 2y ago
A coworker came up with a similar idea: We started a VM using Lima, then ran ps to see what args it passed to QEMU. It was enlightening!
freedomben 2y ago
I agree but I think you meant "Virt Manager"? afaik libvirt is an API, not even a CLI and definitely not a full GUI. For a CLI, virsh is your guy.
- khrbtxyz 2y ago
  There's also virt-install, which is part of the Virt Manager package. https://github.com/virt-manager/virt-manager/blob/main/man/virt-install.rst https://github.com/virt-manager/virt-manager/blob/main/man/v... curl --output-dir $HOME/.local/share/libvirt/images/ -LO https://cloud.debian.org/images/cloud/bookworm/20240507-1740/debian-12-genericcloud-arm64-20240507-1740.qcow2 virt-install --import --osinfo debian12 --disk size=20,backing_store=$HOME/.local/share/libvirt/images/debian-12-genericcloud-arm64-20240507-1740.qcow2 --controller type=scsi,model=virtio-scsi --cloud-init

Neywiny 2y ago

I implemented a bit of a STM32 and it was a chore and a half. I've noticed 2 things with the code base: 1. It's C but they really want C++. Qemu wrote their own class system, foreach loops, containers, etc. And because of that, when I tried to use actual cpp, compilation failed due to how many reserved keywords were used in headers and other mess. 2. As noted in other comments, copy paste and modify. It got me where I need to go. But it was a slog. Eventually I had gdb debugging my qemu build, and gdb debugging the program I was running. I could even connect in from the STM32 IDE which was nice

skrtskrt 2y ago

probably just start using Proxmox, as it's a pretty beginner-friendly FOSS hypervisor with extensive docs and forums, and it's largely a wrapper around qemu. Their docs often include equivalent qemu commands for any UI actions. For anything the UI can't do yet, they only give the QEMU command