7 ms·

Intel Virtualization and Apple Silicon

207 points by zdw 5y ago

hk1337 5y ago
Not very good, even in Docker. Just noticed today that I have a python script that takes 4.5 minutes to process on m1 but only 2.5-3 minutes on an intel Mac.
- hda111 5y ago
  What do you mean with Docker? Docker isn't mentioned in the article. Probably your container would run faster when your container image was native arm64 architecture and there wouldn't be emulation.
Underphil 5y ago
This article has a really misleading feel to it. It appears like it was designed to lure clicks through the title, but in fact it's simply suggesting a bare-metal hypervisor on another machine.
- cromka 5y ago
  And a hackintoshed one, at that!
mschuster91 5y ago
For those thinking they can use a cheap old Mac Pro 4.1/5.1 or other old 2009-2012-ish machines as an ESXi setup which is exactly what I attempted to do... don't if you need anything involving PCIe passthrough - the Intel 5520 chipset has a hardware bug in interrupt remapping.
- EricE 5y ago
  Yikes - that's good to know!
NKosmatos 5y ago
Came here looking for a way to run Windows VMs on Apple silicon (Parallels and VMware can’t boot old win7/10 images on M1 ARM chips) and instead the solution was to use a second PC to run the VMs (with ESXi) and serve these to the brand new M1 based Mac. Oh well, we’ll have to wait until Parallels/VMware sort out the emulation and we’ll be able to run x86 bases OSes at a decent speed on M1.
flyinprogrammer 5y ago
Intel NUCs are now Apple's new biggest dongle!
alophawen 5y ago
> it may never be feasible to do so on Apple Silicon at a useful speed What a uninspiring and uninteresting start of the article.
neonsunset 5y ago
Using Clover is incorrect. It is outdated, full of hacks and nowadays based on OpenCore anyway. However, the right way to go about putting macOS on custom hardware is using OpenCore directly. Depending on your hardware, you may actually get native level of experience or even sometimes better.
hda111 5y ago
It’s much cheaper to install Linux and libvirt on an old PC and then use virt-manager as the client on the Mac. This has no annual license fee.
musicale 5y ago
> At least now I can comfortably work on Apple Silicon without losing my VMware library. Adding an intel NUC to a MacBook Air probably increases the weight – and reduces the time you can run your VMs on battery.
jwr 5y ago
This is pure gold, thank you! I've been wondering what to do about my VMware machines. Some run various windows apps, and one runs Aperture, the only way to access my photo library after Apple screwed me royally by abandoning Aperture. I didn't know what ESXi was.
- EricE 5y ago
  Yes, I wish Apple would sell Aperture to someone who will pick it back up. There still is nothing remotely like it.
- haxxorfreak 5y ago
  If you only need to run Aperture I've had good success using Retroactive[1] on my M1 Max machine running Monterrey to patch it to run under Rosetta. Almost everything works fine except anything tied to iCloud and some of the video and slideshow features. It's been great in letting me access my older photos which have adjustments that I don't want to re-create in Lightroom. It will copy over the missing frameworks and modify the app so the OS will run it even though it's unsupported, if you're interested there is a technical breakdown that goes into detail[2]. [1] https://github.com/cormiertyshawn895/Retroactive https://github.com/cormiertyshawn895/Retroactive [2] https://medium.com/@cormiertyshawn895/deep-dive-how-does-retroactive-work-95fe0e5ea49e https://medium.com/@cormiertyshawn895/deep-dive-how-does-ret...
  - EricE 5y ago
    Nice - thanks for sharing this!
  - jwr 5y ago
    Alas, while Retroactive is fantastic, it doesn't solve the problem of Aperture, because RAW decoding doesn't seem to work. My TIFF and JPEG files are visible, but RAWs show up as black previews and can't be decoded.
kube-system 5y ago
Running your VMs on another machine is not just good for virtualizing Intel while using an Apple Silicon machine -- it's also awesome just to offload resource utilization to another machine. Throw Wireguard on your home network and you can work at the coffee shop all day without plugging in to charge... and you can close your laptop and your long-running tasks keep going.
- 2143 5y ago
  I'm always paranoid about things catching fire though. I rarely leave anything turned on when I leave the house.
- colordrops 5y ago
  What's your recommended wireguard setup?
  - CharlesW 5y ago
    As a happy user, I can highly recommend trying Tailscale. https://tailscale.com/kb/1086/tailscale-vs-wireguard/ https://tailscale.com/kb/1086/tailscale-vs-wireguard/
  - kube-system 5y ago
    Vanilla WireGuard is not hard to use at all, it's by far the easiest to configure VPN software. Some routers have it built in, and some have packages to install it, and the official GUI clients work well: https://www.wireguard.com/install/ https://www.wireguard.com/install/ If you can't run it on your router, you can just set it up on a linux VM and forward a port as necessary. The configuration file is quite simple once you get the hang of it, you've basically just got: 1. a public/private key that you exchange between connected machines 2. the address of the tunnel, locally 3. the address/port to get to the other end 4. what traffic you want to send over the tunnel
    - daemoens 5y ago
      An easier method is to just use PiVPN https://www.pivpn.io https://www.pivpn.io. It's much faster to setup and easier to manage with extremely simple commands.
herpderperator 5y ago
> One advantage very specific to the low-level way ESXi works, because there's no underlying operating system to get in the way, is the ability to 'pass through' real hardware to the virtual machine — say a specific network card, or a USB hub. Or most importantly for macOS: a GPU. This is not at all specific to ESXi... you can do PCIe and USB passthrough with qemu running on Linux. The operating system doesn't get in the way.
- jsolson 5y ago
  Quibbling a bit with the details here. > This is not at all specific to ESXi... you can do PCIe and USB passthrough with qemu running on Linux. This is true. > The operating system doesn't get in the way. This isn't, really, at least not qemu does quite a bit of fiddling under the hoot -- without VFIO or legacy KVM pass-through, the operating system very much gets in the way. Linux now provides facilities that allow qemu to pass through devices -- that is, it provides the APIs necessary to move it back out of the way. I don't know to what extent ESXi looks like pure-kernel setup of passthrough vruss KVM_ASSIGN_DEVICE versus VFIO -- that would be quite interesting.
  - Melatonic 5y ago
    GPU passthrough specifically is tricky - I know ESXi does it well but you of course have to dedicate a physical GPU to a single VM. No sharing. I know they were working on it with Hyper-V awhile back but who knows howe well it works
amelius 5y ago
Why is Apple virtualizing Intel, but is Intel not virtualizing Apple hardware?
- gruez 5y ago
  Intel is incumbent, so most software (short of ios/android apps) are already written/complied for x86. Because of this, there's no real need for intel/amd to support ARM.
- smoldesu 5y ago
  1. Smaller companies like Corellium have already tried making that mistake, and Apple sends their lawyer death squads to their doorstep every time they try pushing a new feature. 2. There aren't really any killer apps for ARM yet. ARM itself is a fine ISA, but the majority of consumer desktop software and server software is still x86 first (particularly on platforms that Intel sells for). Until that changes, I doubt Intel will really care all that much. When it does change, I'm inclined to believe that the rest of the industry will be eyeing even more minimal ISAs like RISC-V. That's mostly just speculation though.
- kmeisthax 5y ago
  Well, first off, Apple isn't virtualizing Intel, they're emulating it. "Virtualization" implies that your CPU has special hardware support for running a kernel as if it were a regular OS process; which almost never crosses architecture boundaries. Apple does have special support in M1 for certain x86 features like total store order, but most of the work is done in software to translate binaries to ARM64 before they are executed. They just omitted emulating anything beyond Intel macOS user-mode, because that's relatively easy and lightweight[0]. Intel doesn't virtualize Apple hardware because it doesn't make business sense and is questionably legal (see also: Corellium). Apple silicon is just-different-enough from a regular ARM64 design that anything that can actually boot macOS almost certainly exists purely to break the macOS license agreement. [0] Specifically, having all your code neatly packaged into executables and libraries means that you can AOT-compile nearly everything and not have to actually interpret or JIT x86 code. There's also no need to emulate, say, x86 context switches or page table walking; you just stop emulation, run the syscall on the ARM side, and then restart the emulation once that's done. Of course, they still have a JIT, just in case you decide to run a binary that also contains a JIT.
vecplane 5y ago
Please forgive my ignorance - what are the benefits of this approach, as opposed to Remote Desktop into a Windows machine?
- ErneX 5y ago
  The ability to have multiple VMs with many different operating systems.
  - Melatonic 5y ago
    But you could easily do that with ESXi free running on your home desktop, running multiple VM's (lets say Windows, linux, BSD, etc etc) and then RDP from a mobile laptop to any of those VM's.
- lstamour 5y ago
  Not needing to run Windows 24/7 to support non-Windows work - one practical benefit is that you might not need to install as many security patches or they might install faster. Another reason might be to avoid running Windows entirely ;-)
dzhiurgis 5y ago
Is there anything like this, but for docker (compose)?
- flyinprogrammer 5y ago
  Go check out docker-machine, but a fork: https://docs.gitlab.com/runner/executors/docker_machine.html#forked-version-of-docker-machine https://docs.gitlab.com/runner/executors/docker_machine.html... Because Docker stopped maintaining it.
- viraptor 5y ago
  Not sure I understand the question, but you can point your docker socket configuration at an external machine of any architecture. That means you can run compose locally and it build/runs remotely.
- deleted 5y ago
  [deleted]
Spooky23 5y ago
I switched to a "physicalization" model a long time ago. It's hard to scale in a bigger company, but it just makes more sense to me to segment the user experience layer from the random compute layer. I run a dozen servers for a little hobby project that has a few customers. One unexpected cool way this has benefited me is I eliminated the need to lug a laptop anywhere. I have a little tools environment setup in iSH on my iPhone and can quickly troubleshoot most things or perform various tasks on the command line. A few others that haven't been scripted yet I can run by SSHing in.
samwillis 5y ago
This is a great tutorial, thanks. Slight aside, with the incredible performance possible with Rosetta on Apple silicone, what are the chances of using it as part of a virtual machine emulating x86-64 with similar performance? I understand that Rosetta “recompiles” to ARM which would probably be harder with a VM but I think it also does real-time conversion to. I have no idea about he details but could QEMU use Rosetta under the hood for emulation?
- shoo_pl 5y ago
  Literally second paragraph of the article mention QEMU-based app that allows you to virtualise x64 machines on M1 - UTM: https://mac.getutm.app/ https://mac.getutm.app/ The thing is, it only supports windows xp + windows 7, and not 8/10/11 (those are supported as arm64). Apparently the amount of work is massive (so are performance hits for the Windows XP/7 when you run it). It's one thing to emulate apps, another entirely to emulate whole advanced OS.
  - samwillis 5y ago
    Yes, I read the article and was in fact aware of UTM, also noted: > the performance penalty is significant I was asking the genuine question as to if it would be theoretically possible to use Rosetta as part of the emulation for a VM to potentially help it to perform better? For example could a tool be installed within a guest OS that effectively send binaries to Rosetta for translation before execution? It just seems to me that if Apple have done such an incredible job with Rosetta wouldn’t it be brilliant if it was possible to use that within an emulated VM on Apple silicone.
    - olliej 5y ago
      No rosetta doesn't do system level emulation. System vs app level are distinctly different problems. Att the app level rosetta can operate at link time (yes it does AoT, but that's essentially just caching), where it knows things like what things are text regions, what the entry points are, it can assume that the code is "correct" (if your code is buggy natively, then rosetta won't stop it going wrong), etc. System level emulation means rosetta (or whatever) loses that transparency, so can't precompile, and can't make any assumptions about code behaving properly. To get an idea of what the performance impact is you should check the performance of JIT compiled code under rosetta. Rosetta ensures that the JIT will work, but the perf hit is staggering. System level emulation basically means treating everything as being JIT code. I'm sure a system level emulator (Qemu?) could do better than for example just slapping a system emulator mode into rosetta, but that would be because rosetta is optimized for app level emulation.
buildbuildbuild 5y ago
Proxmox is great for this as well, minus VMWare’s optimized “VNC” compression and ability to pass through hardware devices.
- digitallyfree 5y ago
  Proxmox has SPICE which is what I use for VDI instances on my server - it's not as good as native but much better than VNC (in my eyes it performs similarly to RDP and is not as fast as Citrix or Horizon). I don't think it supports Mac guests though, only Windows and Linux.
- evol262 5y ago
  I've never used it, but I guarantee that Proxmox can pass through PCIe devices with VFIO, and USB requires no hardware support at all. vmkernel hardware passthrough has exactly the same requirements (IOMMU support).
  - buildbuildbuild 5y ago
    My statement was vague, apologies. I meant client-side pass through of devices. (such as plugging a USB device into your client, and the remote VM believing it is local) SPICE on Proxmox may support this but I find it incredibly clumsy to configure.
F147H34D 5y ago
I had an intel MacBook that was a good fit for doing reverse engineering work on malware. I suppose this method would be kind of a compromise for those that still want to use Mac system, even know it is incompatible with most of their RE tools/virtual machines. Anyone used Ghirda on a M1 Mac?
- angulardragon03 5y ago
  Plenty of anecdotal evidence to suggest Ghidra works fine on Apple Silicon (I also haven’t personally run up against anything unexpected)
  - lukeh 5y ago
    Yup, it works.
- saagarjha 5y ago
  I’ve been using Ghidra on Apple silicon even before M1 came out, it works quite well. I’d suggest building it yourself so you can get native decompilers, though.
  - cyberge99 5y ago
    What Apple Silicon before M1? The Ax Processors?
    - saagarjha 5y ago
      A12Z on the DTK.
Melatonic 5y ago
Surprised this is so new to people - ESXi has been free for years and is the basis for every VMware setup out there. People have been virtualizing MacOS since it was called OSX :-) Using VMware Fusion in this way does sound useful in that it sort of replicates the experience you get with vSphere (for managing the VM's) which is really only something corporate clients would be using and is definitely not free. That being said you could really do this with almost any hypervisor running on a machine at home (or in the cloud, or anywhere) and then connect and control those VM's from a laptop with internet. RDP, VNC, Teamviewer, Citrix, Horizon, whatever - it is just basically us coming full circle back to the dumb Terminal to Mainframe connection but over the internet.
- irq 5y ago
  > Surprised this is so new to people - ESXi has been free for years > Using VMware Fusion in this way does sound useful in that it sort of replicates the experience you get with vSphere VMware's first big product, long before ESX (and later ESXi), was VMware Workstation, released in 1999, which provided x86 virtualization and ran on (and virtualized) both Windows and Linux. It was the first commercial product to easily enable people to run Windows without having to leave their Linux environment. VMware Fusion is essentially the same product, just the spiritual successor to that product that runs on macOS (along with a bunch of modern optimizations, of course) going back to 1999.
  - mobilio 5y ago
    I'm using second product - Connectix VirtualPC (for Windows) because at this time this was only that can run on Windows98SE. VMW require NT/2K at this time. But i couldn't migrate to NT platform due project limitations. VPC works flawless in version 4, but for version 5 they require NT platform too.
    - watersb 5y ago
      Connectix VirtualPC started as a Mac product. I installed Windows 95 on a Mac with VirtualPC back in... 1995. All 26 floppies of it. Installation done, it rebooted the virtual box, and I had to go to a meeting. Cane back an hour or so later, and it had just finished booting up. Good times. Then Microsoft bought them, and I think afterwards the Mac version was no longer supported. (PowerPC was a consortium of Motorola, IBM, and Apple, and was made out of silicon... so technically, that would be "Intel Virtualization on Apple Silicon". :-) )
Cadwhisker 5y ago
Does this method allow you to pass-through USB ports? How is it different from running an XPRA server on the remote machine?
oneplane 5y ago
This isn't really all that new, running macOS on non-Apple hardware has been done, including virtualisation. It also is pretty much the default way to do it unless for whatever reason you need to run it locally (i.e. if you don't have a server or if you need local hardware access). At first I thought this as a 10 year old article, but it appears it really was a re-discovery of the same thing you do with KVM, Xen, Hyper-V, ESXi, VirtualBox etc. for years now.
olliej 5y ago
Wait, MS isn't letting you buy windows/arm for AS?
- cyberge99 5y ago
  Correct. Their license currently prohibits it.
GekkePrutser 5y ago
I don't know why they recommend the $100 Fusion Pro? You can do the exact same with VMware Remote Console which is free :)
- lstamour 5y ago
  Good to know. Not sure if this is still relevant but talks about how to use VMware Remote Console with free ESXi: https://derflounder.wordpress.com/2015/04/18/using-vmwares-standalone-remote-console-for-os-x-with-free-esxi/ https://derflounder.wordpress.com/2015/04/18/using-vmwares-s... You can get VMware Remote Console from the App Store: https://apps.apple.com/ca/app/vmware-remote-console/id1230249825?mt=12 https://apps.apple.com/ca/app/vmware-remote-console/id123024... or at https://www.vmware.com/go/download-vmrc https://www.vmware.com/go/download-vmrc
_8j50 5y ago
Vey cool. This has been my deal breaker. I need to play with x32 and x64 assembly all the time. You know what would be awesome though? If qubes os with its Xen stuff allowed this sort of remote access.
- saagarjha 5y ago
  > I need to play with x32 and x64 assembly all the time. This should work fine in Rosetta?
  - _8j50 5y ago
    I meant VM guests not host apps.

207 points by zdw 5y ago

Not very good, even in Docker. Just noticed today that I have a python script that takes 4.5 minutes to process on m1 but only 2.5-3 minutes on an intel Mac.

hda111 5y ago
What do you mean with Docker? Docker isn't mentioned in the article. Probably your container would run faster when your container image was native arm64 architecture and there wouldn't be emulation.

Underphil 5y ago

This article has a really misleading feel to it. It appears like it was designed to lure clicks through the title, but in fact it's simply suggesting a bare-metal hypervisor on another machine.

cromka 5y ago
And a hackintoshed one, at that!

mschuster91 5y ago

For those thinking they can use a cheap old Mac Pro 4.1/5.1 or other old 2009-2012-ish machines as an ESXi setup which is exactly what I attempted to do... don't if you need anything involving PCIe passthrough - the Intel 5520 chipset has a hardware bug in interrupt remapping.

EricE 5y ago
Yikes - that's good to know!

NKosmatos 5y ago

Came here looking for a way to run Windows VMs on Apple silicon (Parallels and VMware can’t boot old win7/10 images on M1 ARM chips) and instead the solution was to use a second PC to run the VMs (with ESXi) and serve these to the brand new M1 based Mac. Oh well, we’ll have to wait until Parallels/VMware sort out the emulation and we’ll be able to run x86 bases OSes at a decent speed on M1.

flyinprogrammer 5y ago

Intel NUCs are now Apple's new biggest dongle!

alophawen 5y ago

> it may never be feasible to do so on Apple Silicon at a useful speed What a uninspiring and uninteresting start of the article.

neonsunset 5y ago

Using Clover is incorrect. It is outdated, full of hacks and nowadays based on OpenCore anyway. However, the right way to go about putting macOS on custom hardware is using OpenCore directly. Depending on your hardware, you may actually get native level of experience or even sometimes better.

hda111 5y ago

It’s much cheaper to install Linux and libvirt on an old PC and then use virt-manager as the client on the Mac. This has no annual license fee.

musicale 5y ago

> At least now I can comfortably work on Apple Silicon without losing my VMware library. Adding an intel NUC to a MacBook Air probably increases the weight – and reduces the time you can run your VMs on battery.

jwr 5y ago

This is pure gold, thank you! I've been wondering what to do about my VMware machines. Some run various windows apps, and one runs Aperture, the only way to access my photo library after Apple screwed me royally by abandoning Aperture. I didn't know what ESXi was.

EricE 5y ago
Yes, I wish Apple would sell Aperture to someone who will pick it back up. There still is nothing remotely like it.
haxxorfreak 5y ago
If you only need to run Aperture I've had good success using Retroactive[1] on my M1 Max machine running Monterrey to patch it to run under Rosetta. Almost everything works fine except anything tied to iCloud and some of the video and slideshow features. It's been great in letting me access my older photos which have adjustments that I don't want to re-create in Lightroom. It will copy over the missing frameworks and modify the app so the OS will run it even though it's unsupported, if you're interested there is a technical breakdown that goes into detail[2]. [1] https://github.com/cormiertyshawn895/Retroactive https://github.com/cormiertyshawn895/Retroactive [2] https://medium.com/@cormiertyshawn895/deep-dive-how-does-retroactive-work-95fe0e5ea49e https://medium.com/@cormiertyshawn895/deep-dive-how-does-ret...
- EricE 5y ago
  Nice - thanks for sharing this!
- jwr 5y ago
  Alas, while Retroactive is fantastic, it doesn't solve the problem of Aperture, because RAW decoding doesn't seem to work. My TIFF and JPEG files are visible, but RAWs show up as black previews and can't be decoded.

kube-system 5y ago

Running your VMs on another machine is not just good for virtualizing Intel while using an Apple Silicon machine -- it's also awesome just to offload resource utilization to another machine. Throw Wireguard on your home network and you can work at the coffee shop all day without plugging in to charge... and you can close your laptop and your long-running tasks keep going.

2143 5y ago
I'm always paranoid about things catching fire though. I rarely leave anything turned on when I leave the house.
colordrops 5y ago
What's your recommended wireguard setup?
- CharlesW 5y ago
  As a happy user, I can highly recommend trying Tailscale. https://tailscale.com/kb/1086/tailscale-vs-wireguard/ https://tailscale.com/kb/1086/tailscale-vs-wireguard/
- kube-system 5y ago
  Vanilla WireGuard is not hard to use at all, it's by far the easiest to configure VPN software. Some routers have it built in, and some have packages to install it, and the official GUI clients work well: https://www.wireguard.com/install/ https://www.wireguard.com/install/ If you can't run it on your router, you can just set it up on a linux VM and forward a port as necessary. The configuration file is quite simple once you get the hang of it, you've basically just got: 1. a public/private key that you exchange between connected machines 2. the address of the tunnel, locally 3. the address/port to get to the other end 4. what traffic you want to send over the tunnel
  - daemoens 5y ago
    An easier method is to just use PiVPN https://www.pivpn.io https://www.pivpn.io. It's much faster to setup and easier to manage with extremely simple commands.

herpderperator 5y ago

> One advantage very specific to the low-level way ESXi works, because there's no underlying operating system to get in the way, is the ability to 'pass through' real hardware to the virtual machine — say a specific network card, or a USB hub. Or most importantly for macOS: a GPU. This is not at all specific to ESXi... you can do PCIe and USB passthrough with qemu running on Linux. The operating system doesn't get in the way.

jsolson 5y ago
Quibbling a bit with the details here. > This is not at all specific to ESXi... you can do PCIe and USB passthrough with qemu running on Linux. This is true. > The operating system doesn't get in the way. This isn't, really, at least not qemu does quite a bit of fiddling under the hoot -- without VFIO or legacy KVM pass-through, the operating system very much gets in the way. Linux now provides facilities that allow qemu to pass through devices -- that is, it provides the APIs necessary to move it back out of the way. I don't know to what extent ESXi looks like pure-kernel setup of passthrough vruss KVM_ASSIGN_DEVICE versus VFIO -- that would be quite interesting.
- Melatonic 5y ago
  GPU passthrough specifically is tricky - I know ESXi does it well but you of course have to dedicate a physical GPU to a single VM. No sharing. I know they were working on it with Hyper-V awhile back but who knows howe well it works

amelius 5y ago

Why is Apple virtualizing Intel, but is Intel not virtualizing Apple hardware?

gruez 5y ago
Intel is incumbent, so most software (short of ios/android apps) are already written/complied for x86. Because of this, there's no real need for intel/amd to support ARM.
smoldesu 5y ago
1. Smaller companies like Corellium have already tried making that mistake, and Apple sends their lawyer death squads to their doorstep every time they try pushing a new feature. 2. There aren't really any killer apps for ARM yet. ARM itself is a fine ISA, but the majority of consumer desktop software and server software is still x86 first (particularly on platforms that Intel sells for). Until that changes, I doubt Intel will really care all that much. When it does change, I'm inclined to believe that the rest of the industry will be eyeing even more minimal ISAs like RISC-V. That's mostly just speculation though.
kmeisthax 5y ago
Well, first off, Apple isn't virtualizing Intel, they're emulating it. "Virtualization" implies that your CPU has special hardware support for running a kernel as if it were a regular OS process; which almost never crosses architecture boundaries. Apple does have special support in M1 for certain x86 features like total store order, but most of the work is done in software to translate binaries to ARM64 before they are executed. They just omitted emulating anything beyond Intel macOS user-mode, because that's relatively easy and lightweight[0]. Intel doesn't virtualize Apple hardware because it doesn't make business sense and is questionably legal (see also: Corellium). Apple silicon is just-different-enough from a regular ARM64 design that anything that can actually boot macOS almost certainly exists purely to break the macOS license agreement. [0] Specifically, having all your code neatly packaged into executables and libraries means that you can AOT-compile nearly everything and not have to actually interpret or JIT x86 code. There's also no need to emulate, say, x86 context switches or page table walking; you just stop emulation, run the syscall on the ARM side, and then restart the emulation once that's done. Of course, they still have a JIT, just in case you decide to run a binary that also contains a JIT.

vecplane 5y ago

Please forgive my ignorance - what are the benefits of this approach, as opposed to Remote Desktop into a Windows machine?

ErneX 5y ago
The ability to have multiple VMs with many different operating systems.
- Melatonic 5y ago
  But you could easily do that with ESXi free running on your home desktop, running multiple VM's (lets say Windows, linux, BSD, etc etc) and then RDP from a mobile laptop to any of those VM's.
lstamour 5y ago
Not needing to run Windows 24/7 to support non-Windows work - one practical benefit is that you might not need to install as many security patches or they might install faster. Another reason might be to avoid running Windows entirely ;-)

dzhiurgis 5y ago

Is there anything like this, but for docker (compose)?

flyinprogrammer 5y ago
Go check out docker-machine, but a fork: https://docs.gitlab.com/runner/executors/docker_machine.html#forked-version-of-docker-machine https://docs.gitlab.com/runner/executors/docker_machine.html... Because Docker stopped maintaining it.
viraptor 5y ago
Not sure I understand the question, but you can point your docker socket configuration at an external machine of any architecture. That means you can run compose locally and it build/runs remotely.
deleted 5y ago
[deleted]

Spooky23 5y ago

I switched to a "physicalization" model a long time ago. It's hard to scale in a bigger company, but it just makes more sense to me to segment the user experience layer from the random compute layer. I run a dozen servers for a little hobby project that has a few customers. One unexpected cool way this has benefited me is I eliminated the need to lug a laptop anywhere. I have a little tools environment setup in iSH on my iPhone and can quickly troubleshoot most things or perform various tasks on the command line. A few others that haven't been scripted yet I can run by SSHing in.

samwillis 5y ago

This is a great tutorial, thanks. Slight aside, with the incredible performance possible with Rosetta on Apple silicone, what are the chances of using it as part of a virtual machine emulating x86-64 with similar performance? I understand that Rosetta “recompiles” to ARM which would probably be harder with a VM but I think it also does real-time conversion to. I have no idea about he details but could QEMU use Rosetta under the hood for emulation?

shoo_pl 5y ago
Literally second paragraph of the article mention QEMU-based app that allows you to virtualise x64 machines on M1 - UTM: https://mac.getutm.app/ https://mac.getutm.app/ The thing is, it only supports windows xp + windows 7, and not 8/10/11 (those are supported as arm64). Apparently the amount of work is massive (so are performance hits for the Windows XP/7 when you run it). It's one thing to emulate apps, another entirely to emulate whole advanced OS.
- samwillis 5y ago
  Yes, I read the article and was in fact aware of UTM, also noted: > the performance penalty is significant I was asking the genuine question as to if it would be theoretically possible to use Rosetta as part of the emulation for a VM to potentially help it to perform better? For example could a tool be installed within a guest OS that effectively send binaries to Rosetta for translation before execution? It just seems to me that if Apple have done such an incredible job with Rosetta wouldn’t it be brilliant if it was possible to use that within an emulated VM on Apple silicone.
  - olliej 5y ago
    No rosetta doesn't do system level emulation. System vs app level are distinctly different problems. Att the app level rosetta can operate at link time (yes it does AoT, but that's essentially just caching), where it knows things like what things are text regions, what the entry points are, it can assume that the code is "correct" (if your code is buggy natively, then rosetta won't stop it going wrong), etc. System level emulation means rosetta (or whatever) loses that transparency, so can't precompile, and can't make any assumptions about code behaving properly. To get an idea of what the performance impact is you should check the performance of JIT compiled code under rosetta. Rosetta ensures that the JIT will work, but the perf hit is staggering. System level emulation basically means treating everything as being JIT code. I'm sure a system level emulator (Qemu?) could do better than for example just slapping a system emulator mode into rosetta, but that would be because rosetta is optimized for app level emulation.

buildbuildbuild 5y ago

Proxmox is great for this as well, minus VMWare’s optimized “VNC” compression and ability to pass through hardware devices.

digitallyfree 5y ago
Proxmox has SPICE which is what I use for VDI instances on my server - it's not as good as native but much better than VNC (in my eyes it performs similarly to RDP and is not as fast as Citrix or Horizon). I don't think it supports Mac guests though, only Windows and Linux.
evol262 5y ago
I've never used it, but I guarantee that Proxmox can pass through PCIe devices with VFIO, and USB requires no hardware support at all. vmkernel hardware passthrough has exactly the same requirements (IOMMU support).
- buildbuildbuild 5y ago
  My statement was vague, apologies. I meant client-side pass through of devices. (such as plugging a USB device into your client, and the remote VM believing it is local) SPICE on Proxmox may support this but I find it incredibly clumsy to configure.

F147H34D 5y ago

I had an intel MacBook that was a good fit for doing reverse engineering work on malware. I suppose this method would be kind of a compromise for those that still want to use Mac system, even know it is incompatible with most of their RE tools/virtual machines. Anyone used Ghirda on a M1 Mac?

angulardragon03 5y ago
Plenty of anecdotal evidence to suggest Ghidra works fine on Apple Silicon (I also haven’t personally run up against anything unexpected)
- lukeh 5y ago
  Yup, it works.
saagarjha 5y ago
I’ve been using Ghidra on Apple silicon even before M1 came out, it works quite well. I’d suggest building it yourself so you can get native decompilers, though.
- cyberge99 5y ago
  What Apple Silicon before M1? The Ax Processors?
  - saagarjha 5y ago
    A12Z on the DTK.

Melatonic 5y ago

Surprised this is so new to people - ESXi has been free for years and is the basis for every VMware setup out there. People have been virtualizing MacOS since it was called OSX :-) Using VMware Fusion in this way does sound useful in that it sort of replicates the experience you get with vSphere (for managing the VM's) which is really only something corporate clients would be using and is definitely not free. That being said you could really do this with almost any hypervisor running on a machine at home (or in the cloud, or anywhere) and then connect and control those VM's from a laptop with internet. RDP, VNC, Teamviewer, Citrix, Horizon, whatever - it is just basically us coming full circle back to the dumb Terminal to Mainframe connection but over the internet.

irq 5y ago
> Surprised this is so new to people - ESXi has been free for years > Using VMware Fusion in this way does sound useful in that it sort of replicates the experience you get with vSphere VMware's first big product, long before ESX (and later ESXi), was VMware Workstation, released in 1999, which provided x86 virtualization and ran on (and virtualized) both Windows and Linux. It was the first commercial product to easily enable people to run Windows without having to leave their Linux environment. VMware Fusion is essentially the same product, just the spiritual successor to that product that runs on macOS (along with a bunch of modern optimizations, of course) going back to 1999.
- mobilio 5y ago
  I'm using second product - Connectix VirtualPC (for Windows) because at this time this was only that can run on Windows98SE. VMW require NT/2K at this time. But i couldn't migrate to NT platform due project limitations. VPC works flawless in version 4, but for version 5 they require NT platform too.
  - watersb 5y ago
    Connectix VirtualPC started as a Mac product. I installed Windows 95 on a Mac with VirtualPC back in... 1995. All 26 floppies of it. Installation done, it rebooted the virtual box, and I had to go to a meeting. Cane back an hour or so later, and it had just finished booting up. Good times. Then Microsoft bought them, and I think afterwards the Mac version was no longer supported. (PowerPC was a consortium of Motorola, IBM, and Apple, and was made out of silicon... so technically, that would be "Intel Virtualization on Apple Silicon". :-) )

Cadwhisker 5y ago

Does this method allow you to pass-through USB ports? How is it different from running an XPRA server on the remote machine?

oneplane 5y ago

This isn't really all that new, running macOS on non-Apple hardware has been done, including virtualisation. It also is pretty much the default way to do it unless for whatever reason you need to run it locally (i.e. if you don't have a server or if you need local hardware access). At first I thought this as a 10 year old article, but it appears it really was a re-discovery of the same thing you do with KVM, Xen, Hyper-V, ESXi, VirtualBox etc. for years now.

olliej 5y ago

Wait, MS isn't letting you buy windows/arm for AS?

cyberge99 5y ago
Correct. Their license currently prohibits it.

GekkePrutser 5y ago

I don't know why they recommend the $100 Fusion Pro? You can do the exact same with VMware Remote Console which is free :)

lstamour 5y ago
Good to know. Not sure if this is still relevant but talks about how to use VMware Remote Console with free ESXi: https://derflounder.wordpress.com/2015/04/18/using-vmwares-standalone-remote-console-for-os-x-with-free-esxi/ https://derflounder.wordpress.com/2015/04/18/using-vmwares-s... You can get VMware Remote Console from the App Store: https://apps.apple.com/ca/app/vmware-remote-console/id1230249825?mt=12 https://apps.apple.com/ca/app/vmware-remote-console/id123024... or at https://www.vmware.com/go/download-vmrc https://www.vmware.com/go/download-vmrc

_8j50 5y ago

Vey cool. This has been my deal breaker. I need to play with x32 and x64 assembly all the time. You know what would be awesome though? If qubes os with its Xen stuff allowed this sort of remote access.

saagarjha 5y ago
> I need to play with x32 and x64 assembly all the time. This should work fine in Rosetta?
- _8j50 5y ago
  I meant VM guests not host apps.