11 ms·

Virtualizing iOS on Apple Silicon

357 points by walterbell 2y ago

heavyset_go 2y ago
Slightly tangential, but has anyone virtualized ARM macOS on x86-64?
- 3abiton 2y ago
  You should look into the hackintosh project.
  - aspenmayer 2y ago
    https://dortania.github.io/OpenCore-Legacy-Patcher/ https://dortania.github.io/OpenCore-Legacy-Patcher/
    - fl0id 2y ago
      But that’s not arm on x86 is it? My understanding was that it ‘just’ enables things to work on unsupported intel macs, by enabling stuff that still works on newer Intel Macs.
      - aspenmayer 2y ago
        It’s complicated, but you have the right intuition about it. OCLP re-inserts drivers removed by Apple and patches the OS to enable functionality that doesn’t rely on hardware verification or ARM hardware. According to the devs, that’s about all they are currently able to do with current approaches. UTM might do what you want but likely not on x86. https://mac.getutm.app/ https://mac.getutm.app/ > Virtualize macOS as well. > Run multiple instances of macOS on your Apple Silicon Mac with UTM. This can be useful for developers as well as security conscious users. > Note that macOS VM support is limited to ARM based Macs running macOS Monterey or higher.
        tonyedgecombe 2y ago
        >or ARM hardware What is ARM hardware in this case? Did you mean the T2 processor on Intel Macs?
        aspenmayer 2y ago
        I mean both T2 (which I meant by hardware security) and ARM hardware (which means that it relies on either the ARM CPU itself or the way it functions or is implemented). Features like iPhone mirroring apparently rely on hardware support on macOS.
  - amarshall 2y ago
    Hackintosh currently has no way of running ARM-based macOS, so it is of no help here.
- aliher1911 2y ago
  You can try to virtualize generic ARM in qemu and see that it won't reach Raspberry Pi performance. Recent versions should have it available out of the box afaik. Virtualizing Mn cpus would be even less useful.
- grishka 2y ago
  You can't. The term "virtualize" is generally used to mean running an OS via hardware virtualization, where your host CPU natively runs its code but forwards all I/O to a hypervisor. You can only do that with an OS built for the same CPU architecture as your host system. For everything else, like running ARM software on x86 (and vice versa), you'll have to resort to emulation, which involves either interpreting the code or dynamically recompiling it. By definition, you can emulate anything on anything else (someone recently booted Linux for MIPS on an Intel 4004, the first ever microprocessor), but the performance might be a problem.
  - amarshall 2y ago
    TL;DR: emulating any ARM binaries on x86_64 via QEMU is so slow that it is unusable for any general use. This is also less of a QEMU problem and more just that ARM does not emulate well on x86_64 due to their designs.
    - userbinator 2y ago
      I have tried emulating ARM Windows on x86 with QEMU. It is fast enough to see whether something works and not much more (imagine Windows 11 on a 400MHz equivalent processor to understand what the performance was like --- and the host was a fairly recent Intel i7.) ARM Linux is close to usable, however.
      - grishka 2y ago
        It feels like it's just not possible in general to emulate the full instruction set of any CPU with an MMU with an acceptable performance to run modern software. QEMU running Windows for x86 on an M1 isn't very fast either. Only emulating the portion of the instruction set available from the userspace is another story though. At least the way Apple does it with Rosetta and Microsoft with whatever their thing is called, you don't even notice that an app is running under emulation. The only giveaway is that it takes a noticeable time to start for the first time while the code is being translated. It's truly impressive.
ChrisMarshallNY 2y ago
Very cool! I have a feeling that the reason that Apple hasn't made their Simulator into an Emulator, is because they don't want folks digging into the substrate of iOS.
- astrange 2y ago
  Developers still use Intel Macs, and you can't virtualize ARM iOS on that.
  - ChrisMarshallNY 2y ago
    Yeah, I was thinking about the ARM Macs. They are common enough, now, to make it worthwhile.
  - saagarjha 2y ago
    Doesn’t sound like a strong enough reason for the visionOS team.
  - agsnu 2y ago
    The overwhelming majority use ARM Macs these days
- ChocolateGod 2y ago
  Another reason it was a Simulator and not an Emulator to begin with could be because a lot of iOS (or iPhone OS) components at the time were forks of existing Mac OS X libraries.
  - kridsdale1 2y ago
    The reason to begin with was the Mac OS was x86-32 and the iOS environment was arm. Building for intel let the ui devs have high performance by leveraging the existing network stack and graphics compositor. But most of the libraries live parallel in the sim, not using the OS ones. That wouldn’t allow you to simulate different iOS versions.
  - cryptoz 2y ago
cedws 2y ago
According to the author's GitHub profile they are a fresh CS grad - seriously impressive work.
- valval 2y ago
  [flagged]
- lukeh 2y ago
  I imagine a job offer from Apple won't be too far away!
  - ttul 2y ago
    They had better swing this person an offer before they go to the dark side!
  - paulryanrogers 2y ago
    Apple didn't make an offer to the HomeBrew founder. So maybe don't hold your breath.
    - thisislife2 2y ago
      Why would they when Homebrew was amateurish compared to MacPorts built by Apple engineers? (For e.g. it didn't follow accepted unix conventions or macOs conventions on installing libraries for a long time. See this discussion thread https://news.ycombinator.com/item?id=34818192 https://news.ycombinator.com/item?id=34818192 and this comment https://news.ycombinator.com/item?id=34844292 https://news.ycombinator.com/item?id=34844292 for more details). But credit where credit due - the founder is a better marketer than developer and more people know about Homebrew than MacPorts.
    - saagarjha 2y ago
      Apple hired Max Howell to work for them on SPM.
MYEUHD 2y ago
Apple already provides an iOS simulator in XCode. So, what's the benefit of this project over the apple-provided one?
- ykl 2y ago
  The simulator is not actually running real iOS or the iOS build of your app. Instead, when you run an app in the simulator, your app is being compiled to the current Mac’s native instruction set and links/runs against a set of Mac frameworks and libraries that _simulate_ and in some cases only stub in the expected iOS behavior. So as an example, you can’t just take an iOS binary off of the App Store and run it in the iOS Simulator (especially not on an Intel Mac). You also can’t use the simulator to probe and learn anything about how real iOS works internally, because the simulator isn’t really running full iOS. If you drill down in the simulator’s frameworks far enough you eventually just find yourself back in macOS. Contrast with an emulator, where you are just running the full iOS build identical to the build on a real device. You would in theory be able to run any iOS binary unmodified and probe how the real os works. It’s sort of like the difference between running an app through Wine versus running an app in a Windows VM, except in the case of the simulator it’d be like if you had to custom recompile/link a Windows app first against the Wine environment before being able to run it. If you wanted to study how Windows works internally, there's not much you can learn about that from running Wine, but there is quite a lot you could learn from probing a VM running Windows.
  - lisper 2y ago
    Since you are someone who seems to know what they're doing I hope you'll forgive a random unrelated question: do you happen to know if it's possible to call out to M1 code inside Rosetta2? It seems like this should be possible since Rosetta2 is (supposedly) a transpiler and so it's (supposedly) really running M1 code under the hood, but I haven't been able to figure out a way to call out to native M1 code.
    - ykl 2y ago
      That's a great question. The short answer is: no, you can't, but not necessarily for the reason you might expect. The long answer is: Rosetta 2 is indeed a transpiler generating native arm64 code, but transpiled code running via Rosetta 2 vs. native arm64 code in macOS use two different ABIs. Transpiled Rosetta 2 code uses a arm64-ized version of the System V x64 ABI that contains a direct mapping between x64 and arm64 registers, whereas native arm64 code uses the standard arm64 ABI. There's a lot of magic going on in the Rosetta 2 arm64-ized System V ABI that is necessary to make Rosetta 2 work. Koh Nakagawa's work on reverse engineering Rosetta 2 dives into this topic extensively: https://ffri.github.io/ProjectChampollion/part1/ https://ffri.github.io/ProjectChampollion/part1/ One interesting side effect of this ABI difference comes from modern x64 macOS using AVX2 instructions by default but Rosetta 2 not supporting AVX2. Because Rosetta 2 uses a different ABI than native arm64, code running under Rosetta 2 can't just call into the native arm64 system libraries; for calls to system libraries, Rosetta 2 transpiles from the x64 versions of those as well, which are available on Apple Silicon Macs thanks to the universal binary architecture. In macOS, all of the commonly used system dylibs are pre-linked into a single giant file called the dyld cache. Since the native x64 dyld cache contains AVX2 instructions though, it isn't usable by Rosetta 2, so for when a system library call requires going into the dyld cache, Rosetta 2 ships with a _separate second version_ of the x64 dyld cache that is compiled without AVX2. This is an interesting quirk that has proven to be exceptionally useful for getting newer macOS versions running on older unsupported Macs that have Intel CPUs that are too old to support AVX2.
gorkish 2y ago
This is great; for your next trick, can you please figure out how to install MacOS on an iPad so that we can all finally get the dang computer we want Apple to build?
- makeitdouble 2y ago
  You can start with Windows XP https://www.theverge.com/2024/7/22/24200536/windows-xp-ipad-pc-emulation-utm-se https://www.theverge.com/2024/7/22/24200536/windows-xp-ipad-...
  - walterbell 2y ago
    > It took two and a half hours for my iPad to crawl through installation. Jailbroken Apple M1 iPads with iOS16 can use the iOS hypervisor to run VMs without overheating their devices or waiting hours to boot. Still, we can thank Apple for small mercies like UTM, ashell and iSH. As a science experiment, Apple could silently launch a "VM store" with $100 VMs, accessible only via hidden URL. How badly do Apple customers want to use the iPad hardware they already purchased? Could Apple customers be extorted into paying for VMs? Will anyone ever ship a competitive tablet running Linux?
    - acchow 2y ago
      Apple definitely does not want bad reviews on their iPads because the VMs they are selling are crashing more often than their other offerings. Any product Apple sells for actual $ would have meet Apple’s standards of support and customer service, or they would be deteriorating their Goodwill. Except their charging cables. Apple actively trades goodwill for those margins.
      - saagarjha 2y ago
        You do realize that Apple sells Macs, right?
        lostlogin 2y ago
walterbell 2y ago
> Corellium and their virtual iPhone cloud product (only publicly-available “complete” solution) Corellium won their legal case, allowing them to rent [1] iOS Cloud VMs for security research, https://hn.algolia.com/?query=corellium https://hn.algolia.com/?query=corellium If iOS can be virtualized on Apple Silicon Macbooks, it could reduce demand for commercial iOS virtualization services. Individuals: $400/month Business: $60,000/year [1] https://support.corellium.com/subscriptions/pricing https://support.corellium.com/subscriptions/pricing
- ravetcofx 2y ago
  my god, $4-$8/hr who is paying for these VMs?
  - ronsor 2y ago
    This is cheaper than 8xH100 GPU compute time for AI.
    - sangnoir 2y ago
      This is such a left-field comparison. One H100 costs $25,000, whereas one Macbook Pro/iMac/iOS device costs roughly a tenth of that. It's not at all surprising that it's cheaper to rent something that has CapEx costs 2 orders of magnitude less than that of 8xH100 ($200k for the GPUs alone).
      - hamandcheese 2y ago
        I think the point was a $4-$8/hr VM is pretty small potatoes compared to other common corporate expenses.
        sangnoir 2y ago
        The H100 was a terrible example to support that point because it has a much better (rent vs buy) value proposition.
jamesy0ung 2y ago
Discussion of this is on the nick's funny device emporium Discord server. https://discord.com/invite/4HXCHWhf6r https://discord.com/invite/4HXCHWhf6r
hadad 2y ago
the guy that create qemu-t8030 manage to get springboard running [1] , but doesn't made the code public. Is wonderful if the progress can combined with this one [1] https://mastodon.social/@ntrung03/109712247237110967 https://mastodon.social/@ntrung03/109712247237110967
- hadad 2y ago
  Another qemu attempt is this one [1] , q22-qemu (iphone x) 1. https://www.xia0.sh/2024/03/09/Boot-Newer-iOS-with-QEMU-Step-by-Step/ https://www.xia0.sh/2024/03/09/Boot-Newer-iOS-with-QEMU-Step...
reboot81 2y ago
An early Christmas gift to all clickfarms!
jbverschoor 2y ago
Related: https://worthdoingbadly.com/hv/ https://worthdoingbadly.com/hv/ (Hardware-accelerated virtual machines on jailbroken iPhone 12 / iOS 14.1)
kuharich 2y ago
Past comments: https://news.ycombinator.com/item?id=40219423 https://news.ycombinator.com/item?id=40219423

357 points by walterbell 2y ago

heavyset_go 2y ago

Slightly tangential, but has anyone virtualized ARM macOS on x86-64?

3abiton 2y ago
You should look into the hackintosh project.
- aspenmayer 2y ago
  https://dortania.github.io/OpenCore-Legacy-Patcher/ https://dortania.github.io/OpenCore-Legacy-Patcher/
  - fl0id 2y ago
    But that’s not arm on x86 is it? My understanding was that it ‘just’ enables things to work on unsupported intel macs, by enabling stuff that still works on newer Intel Macs.
    - aspenmayer 2y ago
      It’s complicated, but you have the right intuition about it. OCLP re-inserts drivers removed by Apple and patches the OS to enable functionality that doesn’t rely on hardware verification or ARM hardware. According to the devs, that’s about all they are currently able to do with current approaches. UTM might do what you want but likely not on x86. https://mac.getutm.app/ https://mac.getutm.app/ > Virtualize macOS as well. > Run multiple instances of macOS on your Apple Silicon Mac with UTM. This can be useful for developers as well as security conscious users. > Note that macOS VM support is limited to ARM based Macs running macOS Monterey or higher.
      - tonyedgecombe 2y ago
        >or ARM hardware What is ARM hardware in this case? Did you mean the T2 processor on Intel Macs?
        aspenmayer 2y ago
        I mean both T2 (which I meant by hardware security) and ARM hardware (which means that it relies on either the ARM CPU itself or the way it functions or is implemented). Features like iPhone mirroring apparently rely on hardware support on macOS.
- amarshall 2y ago
  Hackintosh currently has no way of running ARM-based macOS, so it is of no help here.
aliher1911 2y ago
You can try to virtualize generic ARM in qemu and see that it won't reach Raspberry Pi performance. Recent versions should have it available out of the box afaik. Virtualizing Mn cpus would be even less useful.
grishka 2y ago
You can't. The term "virtualize" is generally used to mean running an OS via hardware virtualization, where your host CPU natively runs its code but forwards all I/O to a hypervisor. You can only do that with an OS built for the same CPU architecture as your host system. For everything else, like running ARM software on x86 (and vice versa), you'll have to resort to emulation, which involves either interpreting the code or dynamically recompiling it. By definition, you can emulate anything on anything else (someone recently booted Linux for MIPS on an Intel 4004, the first ever microprocessor), but the performance might be a problem.
- amarshall 2y ago
  TL;DR: emulating any ARM binaries on x86_64 via QEMU is so slow that it is unusable for any general use. This is also less of a QEMU problem and more just that ARM does not emulate well on x86_64 due to their designs.
  - userbinator 2y ago
    I have tried emulating ARM Windows on x86 with QEMU. It is fast enough to see whether something works and not much more (imagine Windows 11 on a 400MHz equivalent processor to understand what the performance was like --- and the host was a fairly recent Intel i7.) ARM Linux is close to usable, however.
    - grishka 2y ago
      It feels like it's just not possible in general to emulate the full instruction set of any CPU with an MMU with an acceptable performance to run modern software. QEMU running Windows for x86 on an M1 isn't very fast either. Only emulating the portion of the instruction set available from the userspace is another story though. At least the way Apple does it with Rosetta and Microsoft with whatever their thing is called, you don't even notice that an app is running under emulation. The only giveaway is that it takes a noticeable time to start for the first time while the code is being translated. It's truly impressive.

ChrisMarshallNY 2y ago

Very cool! I have a feeling that the reason that Apple hasn't made their Simulator into an Emulator, is because they don't want folks digging into the substrate of iOS.

astrange 2y ago
Developers still use Intel Macs, and you can't virtualize ARM iOS on that.
- ChrisMarshallNY 2y ago
  Yeah, I was thinking about the ARM Macs. They are common enough, now, to make it worthwhile.
- saagarjha 2y ago
  Doesn’t sound like a strong enough reason for the visionOS team.
- agsnu 2y ago
  The overwhelming majority use ARM Macs these days
ChocolateGod 2y ago
Another reason it was a Simulator and not an Emulator to begin with could be because a lot of iOS (or iPhone OS) components at the time were forks of existing Mac OS X libraries.
- kridsdale1 2y ago
  The reason to begin with was the Mac OS was x86-32 and the iOS environment was arm. Building for intel let the ui devs have high performance by leveraging the existing network stack and graphics compositor. But most of the libraries live parallel in the sim, not using the OS ones. That wouldn’t allow you to simulate different iOS versions.
- cryptoz 2y ago

cedws 2y ago

According to the author's GitHub profile they are a fresh CS grad - seriously impressive work.

valval 2y ago
[flagged]
lukeh 2y ago
I imagine a job offer from Apple won't be too far away!
- ttul 2y ago
  They had better swing this person an offer before they go to the dark side!
- paulryanrogers 2y ago
  Apple didn't make an offer to the HomeBrew founder. So maybe don't hold your breath.
  - thisislife2 2y ago
    Why would they when Homebrew was amateurish compared to MacPorts built by Apple engineers? (For e.g. it didn't follow accepted unix conventions or macOs conventions on installing libraries for a long time. See this discussion thread https://news.ycombinator.com/item?id=34818192 https://news.ycombinator.com/item?id=34818192 and this comment https://news.ycombinator.com/item?id=34844292 https://news.ycombinator.com/item?id=34844292 for more details). But credit where credit due - the founder is a better marketer than developer and more people know about Homebrew than MacPorts.
  - saagarjha 2y ago
    Apple hired Max Howell to work for them on SPM.

MYEUHD 2y ago

Apple already provides an iOS simulator in XCode. So, what's the benefit of this project over the apple-provided one?

ykl 2y ago
The simulator is not actually running real iOS or the iOS build of your app. Instead, when you run an app in the simulator, your app is being compiled to the current Mac’s native instruction set and links/runs against a set of Mac frameworks and libraries that _simulate_ and in some cases only stub in the expected iOS behavior. So as an example, you can’t just take an iOS binary off of the App Store and run it in the iOS Simulator (especially not on an Intel Mac). You also can’t use the simulator to probe and learn anything about how real iOS works internally, because the simulator isn’t really running full iOS. If you drill down in the simulator’s frameworks far enough you eventually just find yourself back in macOS. Contrast with an emulator, where you are just running the full iOS build identical to the build on a real device. You would in theory be able to run any iOS binary unmodified and probe how the real os works. It’s sort of like the difference between running an app through Wine versus running an app in a Windows VM, except in the case of the simulator it’d be like if you had to custom recompile/link a Windows app first against the Wine environment before being able to run it. If you wanted to study how Windows works internally, there's not much you can learn about that from running Wine, but there is quite a lot you could learn from probing a VM running Windows.
- lisper 2y ago
  Since you are someone who seems to know what they're doing I hope you'll forgive a random unrelated question: do you happen to know if it's possible to call out to M1 code inside Rosetta2? It seems like this should be possible since Rosetta2 is (supposedly) a transpiler and so it's (supposedly) really running M1 code under the hood, but I haven't been able to figure out a way to call out to native M1 code.
  - ykl 2y ago
    That's a great question. The short answer is: no, you can't, but not necessarily for the reason you might expect. The long answer is: Rosetta 2 is indeed a transpiler generating native arm64 code, but transpiled code running via Rosetta 2 vs. native arm64 code in macOS use two different ABIs. Transpiled Rosetta 2 code uses a arm64-ized version of the System V x64 ABI that contains a direct mapping between x64 and arm64 registers, whereas native arm64 code uses the standard arm64 ABI. There's a lot of magic going on in the Rosetta 2 arm64-ized System V ABI that is necessary to make Rosetta 2 work. Koh Nakagawa's work on reverse engineering Rosetta 2 dives into this topic extensively: https://ffri.github.io/ProjectChampollion/part1/ https://ffri.github.io/ProjectChampollion/part1/ One interesting side effect of this ABI difference comes from modern x64 macOS using AVX2 instructions by default but Rosetta 2 not supporting AVX2. Because Rosetta 2 uses a different ABI than native arm64, code running under Rosetta 2 can't just call into the native arm64 system libraries; for calls to system libraries, Rosetta 2 transpiles from the x64 versions of those as well, which are available on Apple Silicon Macs thanks to the universal binary architecture. In macOS, all of the commonly used system dylibs are pre-linked into a single giant file called the dyld cache. Since the native x64 dyld cache contains AVX2 instructions though, it isn't usable by Rosetta 2, so for when a system library call requires going into the dyld cache, Rosetta 2 ships with a _separate second version_ of the x64 dyld cache that is compiled without AVX2. This is an interesting quirk that has proven to be exceptionally useful for getting newer macOS versions running on older unsupported Macs that have Intel CPUs that are too old to support AVX2.

gorkish 2y ago

This is great; for your next trick, can you please figure out how to install MacOS on an iPad so that we can all finally get the dang computer we want Apple to build?

makeitdouble 2y ago
You can start with Windows XP https://www.theverge.com/2024/7/22/24200536/windows-xp-ipad-pc-emulation-utm-se https://www.theverge.com/2024/7/22/24200536/windows-xp-ipad-...
- walterbell 2y ago
  > It took two and a half hours for my iPad to crawl through installation. Jailbroken Apple M1 iPads with iOS16 can use the iOS hypervisor to run VMs without overheating their devices or waiting hours to boot. Still, we can thank Apple for small mercies like UTM, ashell and iSH. As a science experiment, Apple could silently launch a "VM store" with $100 VMs, accessible only via hidden URL. How badly do Apple customers want to use the iPad hardware they already purchased? Could Apple customers be extorted into paying for VMs? Will anyone ever ship a competitive tablet running Linux?
  - acchow 2y ago
    Apple definitely does not want bad reviews on their iPads because the VMs they are selling are crashing more often than their other offerings. Any product Apple sells for actual $ would have meet Apple’s standards of support and customer service, or they would be deteriorating their Goodwill. Except their charging cables. Apple actively trades goodwill for those margins.
    - saagarjha 2y ago
      You do realize that Apple sells Macs, right?
      - lostlogin 2y ago

walterbell 2y ago

> Corellium and their virtual iPhone cloud product (only publicly-available “complete” solution) Corellium won their legal case, allowing them to rent [1] iOS Cloud VMs for security research, https://hn.algolia.com/?query=corellium https://hn.algolia.com/?query=corellium If iOS can be virtualized on Apple Silicon Macbooks, it could reduce demand for commercial iOS virtualization services. Individuals: $400/month Business: $60,000/year [1] https://support.corellium.com/subscriptions/pricing https://support.corellium.com/subscriptions/pricing

ravetcofx 2y ago
my god, $4-$8/hr who is paying for these VMs?
- ronsor 2y ago
  This is cheaper than 8xH100 GPU compute time for AI.
  - sangnoir 2y ago
    This is such a left-field comparison. One H100 costs $25,000, whereas one Macbook Pro/iMac/iOS device costs roughly a tenth of that. It's not at all surprising that it's cheaper to rent something that has CapEx costs 2 orders of magnitude less than that of 8xH100 ($200k for the GPUs alone).
    - hamandcheese 2y ago
      I think the point was a $4-$8/hr VM is pretty small potatoes compared to other common corporate expenses.
      - sangnoir 2y ago
        The H100 was a terrible example to support that point because it has a much better (rent vs buy) value proposition.

jamesy0ung 2y ago

Discussion of this is on the nick's funny device emporium Discord server. https://discord.com/invite/4HXCHWhf6r https://discord.com/invite/4HXCHWhf6r

hadad 2y ago

the guy that create qemu-t8030 manage to get springboard running [1] , but doesn't made the code public. Is wonderful if the progress can combined with this one [1] https://mastodon.social/@ntrung03/109712247237110967 https://mastodon.social/@ntrung03/109712247237110967

hadad 2y ago
Another qemu attempt is this one [1] , q22-qemu (iphone x) 1. https://www.xia0.sh/2024/03/09/Boot-Newer-iOS-with-QEMU-Step-by-Step/ https://www.xia0.sh/2024/03/09/Boot-Newer-iOS-with-QEMU-Step...

reboot81 2y ago

An early Christmas gift to all clickfarms!

jbverschoor 2y ago

Related: https://worthdoingbadly.com/hv/ https://worthdoingbadly.com/hv/ (Hardware-accelerated virtual machines on jailbroken iPhone 12 / iOS 14.1)

kuharich 2y ago

Past comments: https://news.ycombinator.com/item?id=40219423 https://news.ycombinator.com/item?id=40219423