7 ms·

Xnu-QEMU-Arm64: iOS on QEMU

190 points by mmastrac 7y ago

albertzeyer 7y ago
Probably the blog posts are a better starting point: - https://alephsecurity.com/2019/06/25/xnu-qemu-arm64-2/ https://alephsecurity.com/2019/06/25/xnu-qemu-arm64-2/ - https://alephsecurity.com/2019/06/17/xnu-qemu-arm64-1/ https://alephsecurity.com/2019/06/17/xnu-qemu-arm64-1/ - Here some more recent screenshots: https://twitter.com/JonathanAfek/status/1221719094661197825 https://twitter.com/JonathanAfek/status/1221719094661197825
markchristian 7y ago
I really wish the opposite of this (QEMU on iOS) existed. The closest I have been able to find is a build of Bochs that doesn't actually work on my iPad. Someday!
- Operyl 7y ago
  It does! https://github.com/utmapp/UTM https://github.com/utmapp/UTM
  - markchristian 7y ago
    OMG
    - Operyl 7y ago
      It’s far from perfect, but it’s fun to tinker in.
      - cat199 7y ago
        what do you use it for? my understanding is x86 on software arm is pretty slow (though some things are probably fine e.g. text/cli utilities)
        saagarjha 7y ago
        UTM is actually not too horrible on the newer iPads with more RAM once the JIT warms up.
  - ChuckMcM 7y ago
    Wow, if I ran on iOS 9.3 I could get some use out of my old "iPad with Retina Display" (aka Ipad 3).
  - fragmede 7y ago
    Fascinating! Any relation to iSH? https://ish.app/ https://ish.app/
    - tbodt 7y ago
      Different projects. iSH has the advantage of not requiring sideloading. (I wrote iSH)
kodablah 7y ago
I would think there's value in wrapping up all of these steps into an easy-to-use CLI (that check for deps, download things as needed w/ opt-in custom paths, etc). Are there legal concerns for making this too easy, or it is just because this is so young? (I personally toss a `build/main.go` and ask people to `go run ./build`...but OP may prefer Python or whatever).
- jedieaston 7y ago
  There would be, but since one of the steps is opening the launchd binary and patching it using a disassembler, I think that any fully automated build process would have to send binaries along that were patched, which Apple would clamp down on immediately. Similar to how certain files used to build Hackintoshes (in the old days) had to be torrented because they were patched. Maybe someone can automate the patcher and get this going.
  - kelnos 7y ago
    It looks like the offset of the instruction that needs to be patched is known in advance, so there's no need to open it in a GUI editor (for example, to do some searching for the right place). A python script would suffice -- just open the file in binary mode, seek to the correct offset, and write out the new instruction.
    - jonyafek 7y ago
      Thanks, these are great suggestions. The project is young and currently we are prioritizing adding support for UI, SEP and interrupts. We do accept code contributions so please contact if you would like to add support for something like this.
daeken 7y ago
This is really awesome for kernel exploitation and such, but for more practical application testing, I've been working on an alternative solution for the last two months. Apple's iOS Simulator ships x86_64 binaries for all the iOS frameworks, so you need to build your application specifically for the Simulator in order for it to run; obviously a no-go for most cases, e.g. apps from the store. So I wrote a translation layer that does ARM64 emulation of the app, while translating calls from ARM->Native libraries and vice versa. It currently supports most non-Swift apps and a small number of Swift apps; once I have proper Swift support, pretty much everything should Just Work (TM). Planning on releasing by end of month (hopefully) for $100/year, aimed primarily at bug bounty hunters and pentesters, though there's nothing stopping you from just using it to play an iOS-specific game or somesuch. If you're curious, you can see Xcode debugging an ARM64 app from the App Store here -- all of their inspection tools work out of the box! https://twitter.com/daeken/status/1242684576738271233 https://twitter.com/daeken/status/1242684576738271233
- saagarjha 7y ago
  Cool! Is this using something like libffi to translate calls at the application↔system framework boundary?
  - daeken 7y ago
    I wish! Haha. No, it's a custom set of trampoline generators that handle the transition. I have one for Objective-C (where I can use the internal reflection tools to get function signatures; fails for variadic functions, so I wrote my own implementations of those, which live on the ARM side), one for C (I generated a function signature database for every header file on the system; otherwise, it shares the trampoline generator with Obj-C), and one for Swift (this is my WIP right now, where I'm generating a similar function database. By far the hardest part so far, if only because I'm so much less familiar with Swift). There's also a native ARM build of libc++ and libc++abi, so that it doesn't have to cross the ARM<->Native boundary for C++ stuff, and a bunch of custom hooks on both sides.
    - saagarjha 7y ago
eatbitseveryday 7y ago
What about macOS itself?
- q3k 7y ago
  macOS on amd64 works well enough on upstream qemu with a few hacks: https://github.com/kholia/OSX-KVM https://github.com/kholia/OSX-KVM
brabel 7y ago
I tried QEMU on my Linux laptop to try to use a Windows VM for testing some of my projects... it was extremely slow, difficult to setup, and after I restarted my laptop, the VM wouldn't boot anymore... googling the error, I just found some people asking for lots of magic commands to be run, then I was supposed to understand the error and run some more commands appropriately to fix things... I just uninstalled QEMU instead. I get it that this is the Linux experience :D you fix stuff by knowing your CLI... but this makes it a huge time-waster for times when all you want is test your actual product. Is this experience typical for anyone else? Or am I, after all these years using Linux, still challenged on the terminal and need to tough the hell up?
- NullPrefix 7y ago
  >lots of magic commands You didn't bother to read the manual and somehow that's not your fault?
  - brabel 7y ago
    The manual for what? KVM? QEMU? The other 500 commands people mentioned?
    - NullPrefix 7y ago
      Would you be willing to share any of these commands?
- structural 7y ago
  I think most of your issues are equating "QEMU == vmware player" or something like this. It's really not: as a matter of fact, QEMU is just a processor emulator, in software. It's not a full-operating-system VM environment at all and can be (and is) used for things like running a single binary without virtualization. Why is this confusing? What you're really looking for is KVM, which -is- a hypervisor that uses the processor's virtualization instructions to implement a complete virtual machine. And then QEMU has a backend, for x86/x86_64, that uses KVM to accelerate its processor virtualization (and I believe some hardware support as well). Using KVM through QEMU at least gives you a command line interface to do things instead of "write a C program to talk to the kernel API", but this is still far from the level of a tool that a desktop user would want to use. QEMU+KVM is basically at the level of one of the random barely-documented executables that you'd see in the installation of a VM software package. So what should you use here? The next layer up is libvirt, which provides a system-level daemon to talk to (so you're not just running an emulator from a terminal that exits when you close the terminal) and a command shell & configuration files for defining machines, networks, storage, and so on. Still not a great UX, but at least now you're at the level of being able to define and run a machine. Many users are going to manage these things through a wizard-based or graphical workflow instead of a CLI, so there's applications like Boxes that's built into GNOME (https://wiki.gnome.org/Apps/Boxes https://wiki.gnome.org/Apps/Boxes) or virt-manager (https://virt-manager.org/ https://virt-manager.org/) which is desktop-agnostic. Both of these also help with establishing a graphical interface to the virtual machine itself, if necessary.

190 points by mmastrac 7y ago

Probably the blog posts are a better starting point: - https://alephsecurity.com/2019/06/25/xnu-qemu-arm64-2/ https://alephsecurity.com/2019/06/25/xnu-qemu-arm64-2/ - https://alephsecurity.com/2019/06/17/xnu-qemu-arm64-1/ https://alephsecurity.com/2019/06/17/xnu-qemu-arm64-1/ - Here some more recent screenshots: https://twitter.com/JonathanAfek/status/1221719094661197825 https://twitter.com/JonathanAfek/status/1221719094661197825

markchristian 7y ago

I really wish the opposite of this (QEMU on iOS) existed. The closest I have been able to find is a build of Bochs that doesn't actually work on my iPad. Someday!

Operyl 7y ago
It does! https://github.com/utmapp/UTM https://github.com/utmapp/UTM
- markchristian 7y ago
  OMG
  - Operyl 7y ago
    It’s far from perfect, but it’s fun to tinker in.
    - cat199 7y ago
      what do you use it for? my understanding is x86 on software arm is pretty slow (though some things are probably fine e.g. text/cli utilities)
      - saagarjha 7y ago
        UTM is actually not too horrible on the newer iPads with more RAM once the JIT warms up.
- ChuckMcM 7y ago
  Wow, if I ran on iOS 9.3 I could get some use out of my old "iPad with Retina Display" (aka Ipad 3).
- fragmede 7y ago
  Fascinating! Any relation to iSH? https://ish.app/ https://ish.app/
  - tbodt 7y ago
    Different projects. iSH has the advantage of not requiring sideloading. (I wrote iSH)

kodablah 7y ago

I would think there's value in wrapping up all of these steps into an easy-to-use CLI (that check for deps, download things as needed w/ opt-in custom paths, etc). Are there legal concerns for making this too easy, or it is just because this is so young? (I personally toss a `build/main.go` and ask people to `go run ./build`...but OP may prefer Python or whatever).

jedieaston 7y ago
There would be, but since one of the steps is opening the launchd binary and patching it using a disassembler, I think that any fully automated build process would have to send binaries along that were patched, which Apple would clamp down on immediately. Similar to how certain files used to build Hackintoshes (in the old days) had to be torrented because they were patched. Maybe someone can automate the patcher and get this going.
- kelnos 7y ago
  It looks like the offset of the instruction that needs to be patched is known in advance, so there's no need to open it in a GUI editor (for example, to do some searching for the right place). A python script would suffice -- just open the file in binary mode, seek to the correct offset, and write out the new instruction.
  - jonyafek 7y ago
    Thanks, these are great suggestions. The project is young and currently we are prioritizing adding support for UI, SEP and interrupts. We do accept code contributions so please contact if you would like to add support for something like this.

daeken 7y ago

This is really awesome for kernel exploitation and such, but for more practical application testing, I've been working on an alternative solution for the last two months. Apple's iOS Simulator ships x86_64 binaries for all the iOS frameworks, so you need to build your application specifically for the Simulator in order for it to run; obviously a no-go for most cases, e.g. apps from the store. So I wrote a translation layer that does ARM64 emulation of the app, while translating calls from ARM->Native libraries and vice versa. It currently supports most non-Swift apps and a small number of Swift apps; once I have proper Swift support, pretty much everything should Just Work (TM). Planning on releasing by end of month (hopefully) for $100/year, aimed primarily at bug bounty hunters and pentesters, though there's nothing stopping you from just using it to play an iOS-specific game or somesuch. If you're curious, you can see Xcode debugging an ARM64 app from the App Store here -- all of their inspection tools work out of the box! https://twitter.com/daeken/status/1242684576738271233 https://twitter.com/daeken/status/1242684576738271233

saagarjha 7y ago
Cool! Is this using something like libffi to translate calls at the application↔system framework boundary?
- daeken 7y ago
  I wish! Haha. No, it's a custom set of trampoline generators that handle the transition. I have one for Objective-C (where I can use the internal reflection tools to get function signatures; fails for variadic functions, so I wrote my own implementations of those, which live on the ARM side), one for C (I generated a function signature database for every header file on the system; otherwise, it shares the trampoline generator with Obj-C), and one for Swift (this is my WIP right now, where I'm generating a similar function database. By far the hardest part so far, if only because I'm so much less familiar with Swift). There's also a native ARM build of libc++ and libc++abi, so that it doesn't have to cross the ARM<->Native boundary for C++ stuff, and a bunch of custom hooks on both sides.
  - saagarjha 7y ago

eatbitseveryday 7y ago

What about macOS itself?

q3k 7y ago
macOS on amd64 works well enough on upstream qemu with a few hacks: https://github.com/kholia/OSX-KVM https://github.com/kholia/OSX-KVM

brabel 7y ago

I tried QEMU on my Linux laptop to try to use a Windows VM for testing some of my projects... it was extremely slow, difficult to setup, and after I restarted my laptop, the VM wouldn't boot anymore... googling the error, I just found some people asking for lots of magic commands to be run, then I was supposed to understand the error and run some more commands appropriately to fix things... I just uninstalled QEMU instead. I get it that this is the Linux experience :D you fix stuff by knowing your CLI... but this makes it a huge time-waster for times when all you want is test your actual product. Is this experience typical for anyone else? Or am I, after all these years using Linux, still challenged on the terminal and need to tough the hell up?

NullPrefix 7y ago
>lots of magic commands You didn't bother to read the manual and somehow that's not your fault?
- brabel 7y ago
  The manual for what? KVM? QEMU? The other 500 commands people mentioned?
  - NullPrefix 7y ago
    Would you be willing to share any of these commands?
structural 7y ago
I think most of your issues are equating "QEMU == vmware player" or something like this. It's really not: as a matter of fact, QEMU is just a processor emulator, in software. It's not a full-operating-system VM environment at all and can be (and is) used for things like running a single binary without virtualization. Why is this confusing? What you're really looking for is KVM, which -is- a hypervisor that uses the processor's virtualization instructions to implement a complete virtual machine. And then QEMU has a backend, for x86/x86_64, that uses KVM to accelerate its processor virtualization (and I believe some hardware support as well). Using KVM through QEMU at least gives you a command line interface to do things instead of "write a C program to talk to the kernel API", but this is still far from the level of a tool that a desktop user would want to use. QEMU+KVM is basically at the level of one of the random barely-documented executables that you'd see in the installation of a VM software package. So what should you use here? The next layer up is libvirt, which provides a system-level daemon to talk to (so you're not just running an emulator from a terminal that exits when you close the terminal) and a command shell & configuration files for defining machines, networks, storage, and so on. Still not a great UX, but at least now you're at the level of being able to define and run a machine. Many users are going to manage these things through a wizard-based or graphical workflow instead of a CLI, so there's applications like Boxes that's built into GNOME (https://wiki.gnome.org/Apps/Boxes https://wiki.gnome.org/Apps/Boxes) or virt-manager (https://virt-manager.org/ https://virt-manager.org/) which is desktop-agnostic. Both of these also help with establishing a graphical interface to the virtual machine itself, if necessary.