8 ms·
In Praise of QEMU
- password4321 4y agoQuickemu: Quickly create and run optimised Win-10,11/macOS/Linux on Linux 20211008 https://news.ycombinator.com/item?id=28797129 https://news.ycombinator.com/item?id=28797129
- krupan 4y agoThanks, Drew, for calling out the author of QEMU and linking to his sight: Fabrice Bellard https://bellard.org/ https://bellard.org/ I didn't know he had done so much! Incredible!
- gtirloni 4y agoCalling out on what specifically?
- avg_dev 4y agohttps://en.wikipedia.org/wiki/Fabrice_Bellard https://en.wikipedia.org/wiki/Fabrice_Bellard Wow.
- versteegen 4y agoHe mostly seems to list his hobby projects on his website, not his paid work (except https://textsynth.com https://textsynth.com and I don't know what else). E.g. he is CTO of Amarisoft. His actual productivity is greater still.
- anthk 4y agoFrom Bellard the things I like most are the "toy" projects. Qemu, tinyemu and such began like that.
- Uehreka 4y agoHis list of hobby projects includes QEMU and ffmpeg. I'm sure Amarisoft is a fine company, but mentioning them before those two would be burying the lede.
- pookeh 4y agoHe also wrote QuickJs a complete Javascript VM.
- jbirer 4y agoThe QEMU project and it's developers are responsible for making my research and hobby OS development on RISC-V and Aarch64 platforms possible, where I learned so much at a pace I otherwise couldn't. Thanks.
- 0x0203 4y agoAgreed. I'm currently using qemu and gdb from an x64 Linux system to debug an aarch64 interrupt problem I'm having. Not the sort of thing you can easily debug on real hardware. I've done similar things with VMware, but as far as I know, that can't do cross architectures like this. It's pretty amazing.
- zen_1 4y agoYeah embedded debugging is an absolute PITA, even with OpenOCD + JTAG exposing a remote gdb server to the target it's just always been so fragile in my experience. Like reloading a changed executable causing gdb to segfault sometimes, and you need to restart every part of the debug chain (device being debugged, openocd, gdb) in order to get back to working on the problem you were originally trying to solve.
- 0x0203 4y agoI hadn't seen OpenOCD yet, so thanks for that. On the boards I'm currently playing with, while they technically have JTAG, it's pin muxed with pins used for the sd card. So I'd have to build a new u-boot to reconfigure the right pins, and get an adapter to break the right jtag pins out of the sd slot, and figure out booting from emmc or spi since I'm currently booting everything from the sd card... QEMU seemed much easier!
- zen_1 4y agoOpenOCD is a massive godsend for any kind of board bringup/embedded development. While I can't speak for your board specifically, if the pin assignment is software reconfigurable (and is reset to sd-card access on reboot!) and you're not using the SD card once your program is running, you could reconfigure the pins in software and then wait for a JTAG debugger to attach (by having an infinite loop you use openOCD to continue past). Of course if you can debug your issue using QEMU that's indeed much easier, thanks Fabrice!
- knob 4y agoDrew is one of my internet personalities where I just follow and read everything I can. I still have to install sway though!
- betwixthewires 4y agoI started reading drew when he had a Gemini capsule. He ditched it, I was disappointed, but I understand why. I still read everything I come across. I didn't even realize he was the maintainer of sway, I was reading his stuff while using his software and didn't even know it! Try out sway, it's fantastic.
- dinom 4y agoIt's truly awesome and has nearly become synonymous with emulation on Linux. Of course without things like KVM, and assorted accelerators, etc. it would lose a fair bit of luster. We all stand on the shoulders of giants to some degree. Nonetheless, I don't know of another free emulator that even comes close... are there any to speak of?
- dijit 4y agoQEMU is still amazing even without KVM; KVM is just a performance improvement for native platforms. But with QEMU you can emulate extremely exotic CPUs on standard x86's; that's the stuff that Console Emulators struggled with immensely!
- ch_123 4y agoNot to trivialize QEMU (I use it often and I love it) but usually the challenge with console emulators are A) emulating supporting hardware such as graphics chips which are often undocumented and B) achieving cycle accurate emulation for platforms which need that. QEMU emulates relatively standard hardware for each platform. Even for x86, sometimes I find 86box or PCem more desirable for certain older hardware/software vs QEMU.
- whb07 4y agoI ran OSX on QEMU/KVM on a Ryzen emulated to be an IceLake Intel cpu and got close to ~90%+ performance of my native machine according to geekbench. So my QEMU OSX instance got better performance than a Mac Pro max, very well near the top ~5 scores for macs in geekbench. Props
- metadat 4y agoHow did you get the Mac VM created? I tried this once but eventually gave up because whatever path I tried it was challenging at every point to create a macOS VM from scratch.
- Dracophoenix 4y agoWhat version of MacOS? Did your IceLake emulation require OpenCore and modified kexts or did you achieve by some other process? Were there any notable compatibility issues with any specific hardware or software?
- whb07 4y agoTake a look here https://github.com/kholia/OSX-KVM https://github.com/kholia/OSX-KVM. Post install, I use the "Opencore-Boot.sh" script to boot the machine and inside of it modify the cpu "Penryn" to Icelake one assuming you're running a Ryzen one. I did get some warnings from QEMU along the lines of "X register doesnt exist" or something but the machine runs fine and geekbench runs fine as well. Play with the cores/threads settings as well. If you go too high (for me was around 8), it doesn't boot up and just hangs.
- anthk 4y agoVirt-Manager it's a must over Qemu, too. I mean, scripts are nice, but qemu has zillions of options. Also: virsh domxml-to-native qemu-argv --domain "your_wm_name" To get the Qemu launching script if you plain to deploy a prototype over thousands of machines.
- vbezhenar 4y agoWhat’s wrong with scripts anyway? I always call qemu from shell script and don’t understand why would I need more layers for that.
- bitwize 4y agoFor the same reason you don't develop front-end JavaScript without a framework, or write a modern video game without using an engine. Unless you're willing to sign up for added friction and pain, you just don't.
- vbezhenar 4y agoIf all I need is to write 100 lines of code, framework will add friction while vanilla JS is perfectly fine.
- anthk 4y agoWell, sometimes it helps to visually see your device tree setup and then when everything it's perfectly configured, you connect the virsh shell to your KVM domain and you begin to spit out all the Qemu scripts so you can edit them for tweaks not seen under Virt-Manager.
- emmelaich 4y agohttps://github.com/lima-vm/lima https://github.com/lima-vm/lima is also excellent
- haolez 4y agoOne of the best programmers I know always say that QEMU is the greatest piece of C code that he has ever come across. It seems that there is some very cool preprocessor magic as well, but I've never dug any deeper than hearing him out :)
- vegetablepotpie 4y agoAs someone who works with QEMU, I must say QEMU is a shining example of how to do object oriented programming in C. That said, trying to follow the QEMU code using static analysis is impossible. Every function call is a function pointer stored as a parameter in a struct that was set up by a macro called in initialization. The only way I could figure out how things are tied together is with GDB. I feel like using C++ would have been a better language choice.
- beagle3 4y agoThe modern QEMU is magic, but I find the original QEMU even more magical: It did not contain any assembly, and was easily portable to new architectures while still providing (relatively) decent performance -- the new TCG backend delivers much beter performance, at the cost of requiring more porting work for new host architectures. The original QEMU had an implementation of the required machine ops in C, but instead of calling those ops, it memcpy()d their implementation, thus enabling binary translation without interpreter overhead -- basically using the underlying "gcc" as a code-generator at runtime without including it. Pure elegance and/or madness. And it worked very well.
- versteegen 4y agoNice. Copying (concatenating) the opcode implementations into an executable memory buffer is well-known. There are variants but it seems to generally be called inline threading or dynamic superinstructions or the memcpy method [1]. If you can manage to do that in C (probably with the GCC computed-goto extension?) without asm or editing machine code I think it's a very good complexity-performance tradeoff, but unfortunately it doesn't seem to be too popular. It's faster than direct-threaded VMs but only eliminates one part of the instruction dispatch overhead compared to a real JIT (e.g. LuaJIT 2 vs LuaJIT 1) which loads operands directly or inlines them and does register allocation. But once you do inline threading, you have opportunities to eliminate more of the interpretation overhead, and the payoff becomes even greater e.g. [2]. [1] Markku Rossi, Kengatharan Sivalingam, 1996, A Survey of Instruction Dispatch Techniques for Byte-Code Interpreters, https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.53.9849 https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.53.... [2] M. Anton Ertl, David Gregg, 2004, Combining Stack Caching with Dynamic Superinstructions https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.90.4604&rep=rep1&type=pdf https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.90...
- ta988 4y agoFor macos people, that's what your docker runs on.
- solarkraft 4y agoQEMU is great, I'm just annoyed by its UI. I'd much rather have a config file I can point it towards than an endlessly long shell script I can't comfortably use comments in (maybe this is a criticism of shell scripts in general). I guess libvirt would be the "docker compose" front-end, but I remember considering that more complex than would be necessary. I know Proxmox has some QEMU management tools (which I find decent), but they don't seem to be very popular outside of it. One cool QEMU thing I recently discovered is the microvm machine type (https://qemu.readthedocs.io/en/latest/system/i386/microvm.html https://qemu.readthedocs.io/en/latest/system/i386/microvm.ht...), which enables it to spawn VMs just as quickly as Firecracker! It would be nicer yet if there was some way to say "give me a shell in this ephemeral VM" like docker does it, but sadly Kata Containers appear to have dropped docker support and I haven't been able to get it to run with the alternative runtimes.
- anthk 4y agoVirt-Manager (and virsh) has option for both config files and scripts creation.
- troutwine 4y ago> I'd much rather have a config file I can point it towards than an endlessly long shell script I can't comfortably use comments in (maybe this is a criticism of shell scripts in general). I end up designing a lot of CLI tools and this is a design problem I run into a lot with my own stuff: how many flags are too many? Introducing a config file works, but now it makes your tool much less feasible to run in a shell script but, like you say, too many flags makes your tool unwieldy to run anyway. The problem is harder if you have a venerable tool that's always been flag heavy. Do you keep lumping flags in or do you steer the community toward the use of a config file? Or a hybrid?
- rcoveson 4y agoOpenVPN's approach is to have every config key imply a valid config file entry and a long-arg. So if you see the line `tun-mtu 1500` in a config file, you know you could use the arg `--tun-mtu 1500` on the command line, and vice versa; if you see somebody use the CLI arg `--proto udp` you know you could put the line in your config file `proto udp`. I think mysqld has a similar approach, though I don't think it's as complete. I'll bet both of those projects would answer that there's no such thing as a tipping point where you tell users to use a config file. They're just two ways of doing the same thing; use whichever makes sense for your deployment.
- pydry 4y agoI dont know why but many of the problems I have with podman/docker running on M1 seem to stem from QEMU. E.g. problems suspending/unsuspending the mac, problems starting/restarting machines, etc. Digging out logs to figure out what went wrong seems to be a pain too.
- ZeroNilNanNada 4y agoI agree. Few years ago there was an uprise of blog posts explaining how to run Linux as a primary system and virtualize Windows machine with GPU passthrough for gaming. ~ 1 year ago I took a dive and did a lot of reading and attempts to make it work, the final results made me really happy. QEMU/KVM/VFIO is one of the best software combinations I've ever used. Witch CPU isolation and scheduling properly setup my Linux part of machine runs Docker with 25 services, while Windows part happily plays games without stutter at the same time. That said, quite a few gaming companies are anti-virtualization (with reasonings being that mostly those who use it are using it for cheating purposes) and ban your account, without rights to appeal (ex. Rainbow Six). I honestly find this stance blunt and it feels like discrimination. But currently such use of virtualization is really niche, so what one can do?
- sm_ts 4y agoI've been using VFIO and maintaining a guide for a few years*, then I've realized that VFIO kinda works, but it's not a reliable technology, and I moved (back) to dual boot. There are very significant pain points, specifically: 1. if one reserves the video card for VFIO, it won't have any power management; this means that it will run hot while doing nothing; in order to work this around: 1a. first has to battle with X, which has an option not-to-take-over-a-card-but-it-takes-it-over-nonetheless 1b. then one can give exclusive access to the graphic card driver, which can be switched out/in when starting/stopping the VM; this unfortunately works, but not reliably 2. the points above apply to nvidia; AMD is worse, as it hasn't supported soft GPU reset until very recently (I think it was added on 5.19 or so) 2a. this means that one starts the VM, then stops it, and most of the times the card will hang 2b. there resize BAR functionality is not supported by VFIO (at least, last year it wasn't), which means, one loses additional performance (I could be ok with it, as the loss is not significant, but performance losses compound) The problem is that all the points above are not in control of the user; the problems happen at driver level (if, say, there is no reset support, one can't add it out of thin air). If one uses Nvidia, and they're ok with the card running hot all the time, then definitely, VFIO works wonder. But this lead me to abandon VFIO, as I don't want that (and the alternative of the card having a most-of-the-time-malfunctiong driver was not appealing, either). Big shame! I loved VFIO :) * https://github.com/64kramsystem/vga-passthrough https://github.com/64kramsystem/vga-passthrough
- frankjr 4y agoQEMU is fantastic, I use it pretty much every day through UTM on macOS (and it doesn't need any 3rd party kernel extensions because it can take advantage of Apple's Hypervisor framework). I used to just use shell scripts to launch VMs but configuring it via the UI is just much easier, maybe I'm getting old. https://mac.getutm.app https://mac.getutm.app
- otachack 4y agoI always preferred a UI when messing with VMs. It's why I've used VirtualBox for the longest time. Part of my hesitancy on using QEMU has been how command line heavy it is where it has just enough friction that I don't want to devote too much time to it. I'll check out your suggestion!
- mongol 4y agoI never understood the interaction between QEMU and KVM. I use the libvirt tools and it works great, but the internals confuse me.
- nortonham 4y agoQEMU is fantastic. I think for spinning up a VM just to try out something, virtualbox is still a good option because of its gui and its simplicity. QEMU is extremely powerful however, and I'm glad it exists.
- dinom 4y agoIt's a bit of an apples vs. oranges comparison though. AFAIK Virtualbox is a hypervisor for x86-64 only. Qemu, otoh, can emulate a number of different hw platforms.
- drewg123 4y agoQEMU is incredibly helpful when debugging issues very early in OS startup (like before console prints work). This sort of debugging was invaluable when working on FreeBSD NUMA stuff and doing things like backing the vm page array with node-local physical memory. My colleague, who is working on kboot for FreeBSD (to run the FreeBSD bootloader as a linux binary that kexec's into FreeBSD from a linuxboot environment) also found QEMU an invaluable tool. In both cases, I think QEMU probably cut development time by 50% or more.
- avg_dev 4y agoI've either never heard of or never bothered to pay attention to SourceHut. https://sourcehut.org/ https://sourcehut.org/ How did GitHub ever get so big when this exists? Is it because GH predates SourceHut? I'm not clear on when SH came into existence, but they have entries on their blog from 2019. > * Absolutely no tracking or advertising > * All features work without JavaScript > * Many features work without an account > * The fastest & lightest software forge [links to https://forgeperf.org/ https://forgeperf.org/ - which is very impressive] > * 100% free and open source software Edit: here's the actual "forge" site: https://sr.ht/ https://sr.ht/
- Tomte 4y agoI'm a paying customer, so I like SourceHut a lot, but it's easy to see why it won't be nearly as big as GitHub. * GitHub was much earlier * GitHub was backed by venture capital, and is now backed by Microsoft * GitHub doesn't force you to adopt an email-only workflow. Pull request are much maligned in tech circles (following Linus Torvald's rant), but people all over dev teams love them. They are a huge argument why companies don't just stand up a git repo and call it done, devs want all those workflows and issue trackers and stuff. SourceHut is very barebones even where it has them. (The exception would be build actions, where SourceHut has no real integrations into big tools, but the core build action system is superior to GitHub's, IMO). Incidentally, the build system depends on qemu: https://drewdevault.com/2018/09/10/Getting-started-with-qemu.html https://drewdevault.com/2018/09/10/Getting-started-with-qemu...
- avg_dev 4y agoI am definitely a big PR user, but I believe that PRs could be implemented as a stand-alone app. That link was very interesting, I read it all. The footnote made me laugh, which is good, because apparently I make poor life choices, and am an embarrassment to us all. ;)
- NelsonMinar 4y agoI recently discovered the magic that is qemu-user-static on an Ubuntu system. Install that and you can instantly run binaries compiled for other architectures. No need for a big VM or anything (*), just run the binary standalone. Combined with a cross-compiler toolchain and you have a nice way to do development for other platforms. (*) the trick being the environment. I'm writing assembly code that just invokes Linux system calls. If you need shared libraries or other OS services, it gets more complicated.