4 ms·

QEMU is thought of as a hardware emulator, but supports "userland" emulation where the processor ISA is emulated but syscalls and memory are translated to the h

by bri3d 7y ago

QEMU is thought of as a hardware emulator, but supports "userland" emulation where the processor ISA is emulated but syscalls and memory are translated to the host OS.

yellowapple 7y ago
I didn't even know QEMU could do that. That's insanely cool; kind of a weird combination of traditional virtualization and Wine.
- kanox 7y ago
  On a distro like debian you can even use it to build-and-run userspace binaries for an unrelated architecture (some chroot magic was required last time I checked).
  - LukeShu 7y ago
    You (1) use binfmt_misc to tell the kernel to use `qemu-ARCHITECTURE` to run binaries for that architecture, then (2) make sure you also have all of the libraries that the binary is linked against, then that binary executable should just run seamlessly. Now, if your ARM binary was compiled to look for libc at /lib/libc.so, but /lib/libc.so is the host's x86 libc, then that obviously won't work; and the easiest way to get the libraries all sorted out is to use a chroot with OS install of the target architecture. If you do go the chroot route, you need to make sure that `qemu-ARCHITECTURE` is statically linked, because it won't have access to the x86 libraries it needs to run after the chroot(2) call happens (which is why most normally-dynamically-linked distros have a "qemu-user-static" package in addition to their normal "qemu-user" package). But with a multilib scheme like Debian's, where all libraries get installed to /lib/ARCHITECTURE-TRIPLET/ instead of /lib/, then it should be possible to install all of the appropriate target libraries on the host system without a chroot! You "should" just need to configure APT to let you install packages built for that architecture. (I haven't actually tried this; I'm not a Debian user, but I am envious of their multilib).
    - nybble41 7y ago
      I've used this to run some 32-bit Linux binaries under Windows Subsystem for Linux (WSL), which only natively supports 64-bit binaries. (Recompilation for 64-bit was not an option.) It wasn't ideal but it did work smoothly for the most part. I just used `dpkg --add-architecture i386 && apt update && apt install libc6:i386` rather than creating a separate chroot. I did have to edit the binfmt registration to remove the 'OC' flags set up by the qemu-user-binfmt package, since these aren't supported by WSL, and manually enable the i386 binfmt which is blocked by default on amd64 platforms. There is also a persistent SIGSEGV in one particular binary which may not be related specifically to running under QEMU.
- LukeShu 7y ago
  One very cool thing that you can do with it is to use binfmt_misc to tell the kernel to use `qemu-arm` to run ARM binaries, then you can chroot in to an ARM device's filesystem from your x86 workstation, and all of the ARM binaries just work.
  - bscphil 7y ago
    I've used this to `apt-get upgrade` a netboot Raspberry Pi installation from the server way faster than you can do it on the Pi itself.
    - leetrout 7y ago
      I was just dealing with this today. QEMU was too slow on my MacBook Air though. Do you have a link to a comprehensive guide on doing this by chance? I was thinking tomorrow I’d just launch an arm instance in AWS and figure it out but I have a dual Xeon workstation at work (windows) that I might try as well.
      - bscphil 7y ago
        I don't have a guide that documents everything I did, but the process is described in pretty good detail here: https://wiki.debian.org/RaspberryPi/qemu-user-static https://wiki.debian.org/RaspberryPi/qemu-user-static You can skip the first part about creating the image, since presumably you already have one. So the process for me is something like: apt-get install qemu qemu-user-static binfmt-support cp /usr/bin/qemu-arm-static ~/rpi_mnt/usr/bin systemd-nspawn -D ~/rpi_mnt bin/bash (Some additional steps needed if you want to use regular chroot instead of nspawn.) Sometimes qemu shows an error saying some operation isn't supported, but this hasn't broken anything yet for me, even after I did a whole Raspbian Stretch -> Buster upgrade this way.
        leetrout 7y ago
        Awesome. Many thanks.
        7y ago
jeffk_teh_haxor 7y ago
The ISA is virtualized (and much faster) if you have KVM installed and your processor supports VTx extensions.

by bri3d 7y ago

QEMU is thought of as a hardware emulator, but supports "userland" emulation where the processor ISA is emulated but syscalls and memory are translated to the host OS.

yellowapple 7y ago

I didn't even know QEMU could do that. That's insanely cool; kind of a weird combination of traditional virtualization and Wine.

kanox 7y ago
On a distro like debian you can even use it to build-and-run userspace binaries for an unrelated architecture (some chroot magic was required last time I checked).
- LukeShu 7y ago
  You (1) use binfmt_misc to tell the kernel to use `qemu-ARCHITECTURE` to run binaries for that architecture, then (2) make sure you also have all of the libraries that the binary is linked against, then that binary executable should just run seamlessly. Now, if your ARM binary was compiled to look for libc at /lib/libc.so, but /lib/libc.so is the host's x86 libc, then that obviously won't work; and the easiest way to get the libraries all sorted out is to use a chroot with OS install of the target architecture. If you do go the chroot route, you need to make sure that `qemu-ARCHITECTURE` is statically linked, because it won't have access to the x86 libraries it needs to run after the chroot(2) call happens (which is why most normally-dynamically-linked distros have a "qemu-user-static" package in addition to their normal "qemu-user" package). But with a multilib scheme like Debian's, where all libraries get installed to /lib/ARCHITECTURE-TRIPLET/ instead of /lib/, then it should be possible to install all of the appropriate target libraries on the host system without a chroot! You "should" just need to configure APT to let you install packages built for that architecture. (I haven't actually tried this; I'm not a Debian user, but I am envious of their multilib).
  - nybble41 7y ago
    I've used this to run some 32-bit Linux binaries under Windows Subsystem for Linux (WSL), which only natively supports 64-bit binaries. (Recompilation for 64-bit was not an option.) It wasn't ideal but it did work smoothly for the most part. I just used `dpkg --add-architecture i386 && apt update && apt install libc6:i386` rather than creating a separate chroot. I did have to edit the binfmt registration to remove the 'OC' flags set up by the qemu-user-binfmt package, since these aren't supported by WSL, and manually enable the i386 binfmt which is blocked by default on amd64 platforms. There is also a persistent SIGSEGV in one particular binary which may not be related specifically to running under QEMU.
LukeShu 7y ago
One very cool thing that you can do with it is to use binfmt_misc to tell the kernel to use `qemu-arm` to run ARM binaries, then you can chroot in to an ARM device's filesystem from your x86 workstation, and all of the ARM binaries just work.
- bscphil 7y ago
  I've used this to `apt-get upgrade` a netboot Raspberry Pi installation from the server way faster than you can do it on the Pi itself.
  - leetrout 7y ago
    I was just dealing with this today. QEMU was too slow on my MacBook Air though. Do you have a link to a comprehensive guide on doing this by chance? I was thinking tomorrow I’d just launch an arm instance in AWS and figure it out but I have a dual Xeon workstation at work (windows) that I might try as well.
    - bscphil 7y ago
      I don't have a guide that documents everything I did, but the process is described in pretty good detail here: https://wiki.debian.org/RaspberryPi/qemu-user-static https://wiki.debian.org/RaspberryPi/qemu-user-static You can skip the first part about creating the image, since presumably you already have one. So the process for me is something like: apt-get install qemu qemu-user-static binfmt-support cp /usr/bin/qemu-arm-static ~/rpi_mnt/usr/bin systemd-nspawn -D ~/rpi_mnt bin/bash (Some additional steps needed if you want to use regular chroot instead of nspawn.) Sometimes qemu shows an error saying some operation isn't supported, but this hasn't broken anything yet for me, even after I did a whole Raspbian Stretch -> Buster upgrade this way.
      - leetrout 7y ago
        Awesome. Many thanks.
      - 7y ago

jeffk_teh_haxor 7y ago

The ISA is virtualized (and much faster) if you have KVM installed and your processor supports VTx extensions.