19 ms·

About the Rosetta Translation Environment

221 points by loun 6y ago

gjsman-1000 6y ago
The ability to switch a universal binary between running x86 and ARM versions so as to support non-updated x86 plugins is really cool.
dsabanin 6y ago
From the page: What Can't Be Translated? Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. However, Rosetta doesn’t translate the following executables: * Kernel extensions * Virtual Machine apps that virtualize x86_64 computer platforms Some people are not going to be happy about this. Edit: But I personally am okay with that.
- yjftsjthsd-h 6y ago
  Like... it's not ideal, maybe, but kernel extensions are already a little touchy so I'm not surprised that they at least need to recompile, and VMs likewise are unfortunate but they can just rebuild and be good (i.e. I'm betting less than a month before people are running around using qemu-system-x86_64 natively built for ARM again).
- unix_fan 6y ago
  Meh, kernel extensions are on life support at this point.
- perardi 6y ago
  Apple wants kexts gone. They announced the deprecation last year, and this is another chance to force the change. (And for good reasons, if my layman's understanding of the security is at all correct.) [https://developer.apple.com/support/kernel-extensions/](https://developer.apple.com/support/kernel-extensions/) https://developer.apple.com/support/kernel-extensions/](http...
  - selsta 6y ago
    Apple ARM will support kexts. https://developer.apple.com/documentation/apple_silicon/installing_a_custom_kernel_extension https://developer.apple.com/documentation/apple_silicon/inst...
    - perardi 6y ago
      Well color me surprised. I figured that’d be on the chopping block.
      - monocasa 6y ago
        I was surprised too. I wonder which crazy peripheral (or maybe it was antivirus/intrusion detection?) made them not go all in on "only apple gets to be in kernel space".
        perardi 6y ago
        Judging by this horrible Bitdefender Endpoint Security that IT has foisted upon me: device management and anti-malware stuff.
        saagarjha 6y ago
        Maybe Dropbox?
marcinzm 6y ago
>However, Rosetta doesn’t translate the following executables: Virtual Machine apps that virtualize x86_64 computer platforms How will this impact docker? Does this mean you can't run x86 docker containers on new Apple laptops?
- cloogshicer 6y ago
  Not sure about the details for how this works, but they specifically mentioned Docker to be supported at the keynote.
  - marcinzm 6y ago
    x86 docker or just docker? There's ARM docker containers as well.
  - dividedbyzero 6y ago
    Docker already runs fine on ARM64, like on Raspberry Pis, so there would be no Rosetta translation needed for ARM64 images. If Rosetta could translate binaries in x86/AMD64 images, that would be very helpful indeed. Likewise, will it support CLI programs at all?
- EtienneK 6y ago
  I would assume you’d only be able to run ARM Docker containers.
  - marcinzm 6y ago
    Which is less than ideal if you're using x86 servers since you no longer have the same dev containers as production containers.
    - EtienneK 6y ago
      Yeah. I predict a sudden increase of ARM servers as well.
tomduncalf 6y ago
Are the limitations on x86_64 virtualization likely to be for technical reasons, or patent reasons? I read a comment on here alluding to some patents on x86_64 virtualization expiring later this year: https://news.ycombinator.com/item?id=23612256 https://news.ycombinator.com/item?id=23612256 - could that mean that there is a chance this might happen and they are keeping it quiet for now, or are patents likely unrelated?
- unicornfinder 6y ago
  If I had to guess it's because Rosetta isn't fully emulating an x86_64 instruction set but rather dynamically translating API calls for the most part.
  - mkl 6y ago
    I think it's neither of those things. The x86_64 machine code instructions are translated into ARM instructions, so there's no emulation and the API calls don't need to change.
    - monocasa 6y ago
      > The x86_64 machine code instructions are translated into ARM instructions, so there's no emulation That's what emulation is.
      - mkl 6y ago
        I think emulation would be interpreting the instructions one by one and doing what they say in software (a bit like Python's byte code interpreter). Rosetta translates the binary to another (now native) binary, and then runs it like normal, and much of the time this will be an ahead-of-time once off translation.
        monocasa 6y ago
        Recompilation is a valid emulation strategy. You're just amortizing client instruction decode by doing it ahead of time. Last time I checked, that's how RPCS3 worked for instance, since PS3's GameOS didn't generally allow JITs anyway.
underdeserver 6y ago
Translating JITted code is a pretty cool trick.
- saagarjha 6y ago
  At runtime it doesn't really matter where you code is coming from, as long as you do flushes at the right spots.
  - jeffbee 6y ago
    But doesn't this mean that they are translating and running the JIT and the JIT is creating new executable x86 code which then also needs to be translated? I'm a bit baffled as to how any of this can work properly, for example if my JIT is sampling instructions to determine when to optimize, does Rosetta need to reverse-translate the actual instruction pointer back to the original x86 code offset?
    - underdeserver 6y ago
      > But doesn't this mean that they are translating and running the JIT and the JIT is creating new executable x86 code which then also needs to be translated? Yup. > I'm a bit baffled as to how any of this can work properly, for example if my JIT is sampling instructions to determine when to optimize, does Rosetta need to reverse-translate the actual instruction pointer back to the original x86 code offset? Well, not exactly. Let's take JS as an example. First of all, you're only sampling calls that are still in the native language, so your instruction pointer is in the emulated virtual machine itself anyway, which is agnostic to the CPU architecture. See [1] for where SpiderMonkey samples. What happens then is that the engine sees a function being called many times, so the JIT compiler decides to compile a function; it compiles the function and writes the compiled function to memory, then overwrites that function's address with a native function pointer. The easiest hook the OS has to this process is mprotect(2). In modern OSes, memory is generally either writable or executable, but not both; so if you want to compile and write, and then execute, you need to call mprotect() between those steps to set permissions. Knowing that your binary is Rosetta'd, MacOS can assume your JIT code is also AMD64 and translate that page on the fly. All they have to do is make sure the virtual memory addresses are correct - to not mess up those function pointers. They can either do it lazily, through a pagefault handler, or greedily when mprotect is called to add executable permissions to the JIT-allocated memory. [1] https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/JIT_Optimization_Strategies https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Sp...
nickcw 6y ago
I thought this was the most interesting paragraph. > What Can't Be Translated? > Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. I guess translation of JIT compiled stuff implies this isn't a once of translation. I guess translating plugins implies that too. It sounds like very clever stuff to me! > However, Rosetta doesn’t translate the following executables: > > Kernel extensions Fair enough > Virtual Machine apps that virtualize x86_64 computer platforms I guess most VMs rely on hardware virtualization which would be tricky to translate well. > Rosetta translates all x86_64 instructions, but it doesn’t support the execution of some newer instruction sets and processor features, such as AVX, AVX2, and AVX512 vector instructions. If you include these newer instructions in your code, execute them only after verifying that they are available. For example, to determine if AVX512 vector instructions are available, use the sysctlbyname function to check the hw.optional.avx512f attribute. These sound like they should be relatively straight forward to translate. I wonder why they didn't? Lack of time? Or perhaps because translating them means that they don't run fast enough to be useful and the fallback paths are likely to run quicker.
- leeter 6y ago
  RE AVX and above: My guess is that they are licensing the Transmeta IP which doesn't include those licenses IIRC. So unless they wanted to get sued they'd have to be careful. I suspect some SSE instructions may also not be supported as I don't recall if Transmeta had a license up to SSE4.
  - monocasa 6y ago
    I doubt they're licencing Transmeta. Transmeta never had an x86_64 processor.
    - leeter 6y ago
      True, but they did have the license IIRC. Otherwise they'd be licensing from VIA or whomever owns that license now. That license AFAIK would include up to SSE4. It's not like Apple is going to tell us. But the JIT nature makes me think it's Transmeta as the company currently holding that IP is a "Patent enforcement entity".
jayyhu 6y ago
>Rosetta translates all x86_64 instructions, but it doesn’t support the execution of some newer instruction sets and processor features, such as AVX, AVX2, and AVX512 vector instructions. No AVX will probably mean that the vast majority of pro/graphics intensive apps won't work out of the box with Rosetta.
- beervirus 6y ago
  Do those programs blindly try to execute AVX instructions without checking whether they’re supported? What happens if you're just running on an older intel processor?
  - saagarjha 6y ago
    Usually you'd want to check cpuid or similar and fall back to something reasonable.
  - jayyhu 6y ago
    Apple seems to suggest that all well behaving programs: >use the sysctlbyname function to check the hw.optional.avx512f attribute So my guess it they probably just replace the AVX instructions with no-ops and let it blow up if the correct checks/fallbacks aren't in place.
    - monocasa 6y ago
      Or just SIGILL rather than noop.
    - throwaway2048 6y ago
      it would be a lot more normal to trap the illegal instruction, the same thing that happens if you give a CPU a nonsense nonexistent instruction.
  - makomk 6y ago
    All well-behaving software will of course check if AVX is supported before trying to use it, but at least in some cases that just means they'll refuse to install or run on systems that don't have it: https://www.pro-tools-expert.com/production-expert-1/2019/9/5/is-this-software-issue-the-final-curtain-call-for-old-mac-pro-cheese-graters-used-in-todays-recording-studios https://www.pro-tools-expert.com/production-expert-1/2019/9/... (Developing parallel code paths and keeping them working indefinitely is a lot of extra work, and AVX has been around for close to a decade at this point.)
bjpirt 6y ago
Having big flashbacks to the switch from PPC to x86 here. Rosetta worked relatively smoothly during that transition so fingers crossed it will be ok here too. Though with Docker support on the mac already being a second class citizen to running on Linux I wonder if a lot of devs will stop using macs for dev
- LeSaucy 6y ago
  Highly unlikely Apple would cede the software engineering market for a competitor to step in. Almost everyone in my company would prefer to have a MacBook vs a Thinkpad/Dell/etc if given the choice.
  - SomeHacker44 6y ago
    Speaking solely for myself, I do not intend to go with this experiment after using Macs for 16 years and change. I have found that with a few rough edges, Windows 10 and WSL fulfills my dev needs, and except for the Apple apps Logic and Final Cut Pro, virtually all my software is cross platform. Besides, I can use Ableton and Adobe Premiere which I also already "own."
    - jjcon 6y ago
      > I do not intend to go with this experiment I’m sure they lost people when they switched off of ppc too but just like last time Apple is playing the long game. Intel just can’t keep up anymore.
      - PaulDavisThe1st 6y ago
        Intel can't (maybe). AMD can. In fact right now, AMD doesn't even have to keep up, because it's substantially out in front.
        jjcon 6y ago
        AMD has worse thermals than intel does which is likely the reason Apple is switching in the first place
godzillabrennus 6y ago
The Apple transition to ARM for me is sad. There was so much positivity around the Intel transition. It opened up the Mac platform. Now it’s going back into a closed black box.
- Bigpet 6y ago
  I think I'm missing something. How is ARM closing things up? Does Apple have a bunch of processor extensions that make it incompatible with other ARM processors? Or do you think they'll take the chance to kill boot-camp or something?
  - monocasa 6y ago
    They'll almost certainly only allow their signed kernels to run. And that's in addition to not documenting any of the peripherals.
    - larkost 6y ago
      Apple already has a program for select third party companies to get their own signing keys for kernel extensions. They have been tightening that circle over the last few years as they provide more and more generic interfaces so that companies can replace KEXTs with user space code. From what they said in the State of the Union keynote they want to get that down to nothing, but realize they can't do that yet. No details were provided, but there might be more in the KEXT sessions if you are interested.
  - proninyaroslav 6y ago
    I think Apple has more maneuvers on ARM, since in general the PC architecture is open for modifications, unlike ARM. They will do it the same way as with iOS devices - a closed bootloader and hardware protection.
  - qubex 6y ago
    The “ARM world” is almost inconceivably varied by x86 standards. There’s not even an agreed-upon boot procedure (the Raspberrry Pi’s CPU famously gets booted by the GPU). Efforts to standardise this in the server space are only just getting off the ground with the Arm Server Base Boot Requirements version 1.2 being ratified last year. I doubt you’ll find Apple subscribing to anything that they won’t develop in-house. This is, after all, the company that developed the Lightening connector and sticks on its iPhones to it despite the whole industry shifting to USB-C. Basically the OS situation on Macs post-ARM is probably going to look like the Linux situation on iDevices currently: occasionally somebody manages to text-mode-boot an iPhone 7 into a profoundly unstable hacked-up kernel and there is much merriment and rejoicing, but really... macs are going to become rather exotic (on the inside). Have you ever wondered about your iPad’s boot sequence? Because you’ve never had to think about it. Notice how you can’t install versions of i(Pad)OS that are no longer signed? You think Apple will allow booting unsigned images? And all the Secure Enclave infrastructure they’ve built up? Nah.
DCKing 6y ago
Rosetta's goal is to support legacy Mac apps. It's quite finely scoped, but I hope Apple will go above this and make it available as part of their virtualization framework. This way e.g. Parallels and Docker could use this to provide some way to tap in probably the fastest way to run x86 on ARM Macs. This would mean Rosetta would go beyond its stated scope, and certainly go beyond what the relatively short-lived Rosetta 1 did, but would make it easier to integrate ARM Macs in what is still an x86 world. I'm worried they're not going to do this because x86 emulation is probably too slow (people will attempt to run their AAA games in a VM). It would also mean that Apple will need to support Rosetta 2 forever. If they're not going to do this, everybody will have to rely on qemu. Qemu is great, but I hope it will perform adequately for at least basic Docker stuff...
- Angostura 6y ago
  FWIW they showed the latest Tombraider game running on Rosetta on a dev Mac said to be using the ARM chip from the high-end iPad. It ran pretty well. Not sure if I would quite count this as AAA
  - DCKing 6y ago
    I don't think that was running on Rosetta 2's dynamic recompiler, but instead used Rosetta 2's other stated feature of being able to translate existing apps at install time. In the demo they state that they downloaded the game from the Mac App store. Certainly static recompilation yields somewhat-close-to-native performance when it is available, but that is not possible for x86 "VMs".
  - toyg 6y ago
    > It ran pretty well It was a postproduced, recorded promotional video, not a live demo. They could have shown laptops reaching Mars and you’d have thought “that landing gear looks pretty solid”.
  - monocasa 6y ago
    IDK, what I saw was a several year old game running on the lowest settings and still stuttering a bit.
mkl 6y ago
> Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. How on Earth does it do that? If executable code is being generated at runtime, it's going to be x86_64 binary machine code still (there are too many ways to generate valid machine code, and it won't know right away whether you're JITting, or cross compiling and actually want x86_64), so Rosetta would need to detect when the code's about to be run, or when it's marked as executable, and translate the machine code in that bit of memory just in time. The length of the ARM code might be longer, so it would have to be in a different block of memory, with the original x86_64 code replaced with a jump to the new code or something. It's late at night here, so maybe I'm missing a simpler approach, but I'm a bit surprised they have it working reliably enough to make such a general statement (there being a great variety of JIT systems). From a quick search I can't tell if Microsoft's x86-on-ARM translation in Windows 10 ARM supports JITs in the program being run.
- lsllc 6y ago
  It probably just hooks mmap() (or mprotect()) and looks for when the JIT sets PROT_EXEC on memory it hasn't translated yet. Here's a tutorial for a simple JIT that uses mmap() with PROT_EXEC to write machine code to before executing: https://github.com/spencertipping/jit-tutorial https://github.com/spencertipping/jit-tutorial
  - monocasa 6y ago
    Since they don't enforce W^X you need to go a little deeper than that since some JITs reuse the same memory later for new traces. So what you'll do is silently enforce W^X to trap code modifications, but not propagate those traps to the emulated code.
    - lsllc 6y ago
      Right, if you look at the example JIT project I posted, they start by creating the memory with mmap() and PROT_EXEC before generating any code. So yes, you'd need to trap subsequent writes to allow for retranslation [I assume they have hooks in the Darwin kernel for this].
nothis 6y ago
So how fast will this be compared to native? As I understand, it's not "emulation" but something more low level?
- Synaesthesia 6y ago
  Did you ever use PPC apps on Intel through Rosetta? It was a substantial slowdown, like at least 2x. But the app was usable.
rz2k 6y ago
Does anyone have any insight into the business logistics involved in a transition like this? I presume Apple has done maintenance on transitioning desktop OS X to ARM as an option for a very long time. How many years ago would they have had to decide that was the direction they were going to make it reality? 2015? How many people would have been working on it? How many billions of dollars? How does the cost of developing the software compare to the cost of developing a new processor, or compared to tooling a production line for an entirely new platform? I'm really curious about the share of institutional resources and attention this likely involved. I wonder how important it is as context to Apple over the past few years. I also wonder if it heralds a new direction in personal computers. Every time I've considered that my Mac isn't that great, the alternatives aren't that great either. Would Apple ever decide to seriously compete for a dominant share of the personal computer market? And finally, I am also curious about the accounting and financial disclosures required in such decisions. How much are institutional investors informed of long range plans? There's naturally a lot more pressure to distribute cash to shareholders if you don't know about some five-year-plan, yet that pressure waxes and wanes from activist investors, and institutional investors like CalPERS always seemed to be on board with Apple's direction anyway. Do unofficial leaks prevent violations of insider trading rules?
- ryanianian 6y ago
  > How many billions of dollars? Rounding-error-sized marginal costs. Even if you don't plan on pivoting architectures, it's good practice to write as much code as possible without assembly/intrinsics. If you're writing standards-compliant C/C++/Objective-C/Swift/* it's not a huge burden to change architectures. They also have past-experience with such transitions, so hopefully they kept the "keep the code portable" mantra close over the past 10+ years. The choice to do this seems relatively obvious for Apple. They have spent years building up their silicon game with portables. Their frustration with Intel is also palpable, so it was probably "just" a waiting game (not "how" or "if") in deciding when to pull the trigger and start using those chips outside of iPhones/iPads.
  - Someone 6y ago
    If you have to maintain any highly optimized assembly/intrinsics code, it’s a very good idea to also maintain an “as portable as feasible” C version for comparison testing.
andoma 6y ago
I wonder how it handles the stricter memory ordering from x86? For example, this code: void store_value_and_unlock(int *p, int *l, int v) { *p = v; __sync_lock_release(l, 0); } on x86_64 compiles to: mov dword ptr [rdi], edx mov dword ptr [rsi], 0 ret vs on arm64: str w2, [x0] stlr wzr, [x1] ret Notice how the second store on ARM is a store with release semantics to ensure correct memory ordering as it was intended in the original C code. This information is lost as it's not needed on x86 which guarantees (among other things) that stores are visible in-order from all other threads.
- monocasa 6y ago
  That's the big piece I've been wondering about too. Three options as I see it (none of them great): 1) Pin all threads in an x86 process to a single core. You don't have memory model concerns on a single core. 2) Don't do anything? Just rely on apps to use the system provided mutex libraries, and they just break if they try to roll their own concurrency? Seems like exactly the applications you care about (games, pro apps), would be the ones most likely to break. 3) Some stricter memory model in hardware? Seems like that'd go against most of the stated reason for switching to ARM in the first place.
  - andoma 6y ago
    > 3) Some stricter memory model in hardware? Seems like that'd go against most of the stated reason for switching to ARM in the first place. I would assume that a more strict memory model would be enabled only for processes that needs it (ie, Rosetta translated ones). So a cpu-flag is set/cleared when entering/exiting user mode for those processes. Does this require a separate/special cache coherency protocol? A complete L1d flush when entering/leaving these processes (across all CPUs)? Not and expert in this field and it feels complicated for sure. Is it worth it for just emulating "legacy" applications during a transitional period? Perhaps Apple can pull it off though.
    - monocasa 6y ago
      > I would assume that a more strict memory model would be enabled only for processes that needs it (ie, Rosetta translated ones). So a cpu-flag is set/cleared when entering/exiting user mode for those processes. Does this require a separate/special cache coherency protocol? The benefits you'd get from going to a weaker memory model are by not having that extra coherency in the critical path in the first place. Adding extra muxes in front of it to make it optional would be worse than just having it on all the time. > A complete L1d flush when entering/leaving these processes (across all CPUs)? That wouldn't help because two threads could be running at the same time on different cores against their respective L1 and store buffers.
viro 6y ago
So does this mean no windows VMs?
- tonyedgecombe 6y ago
  Yes, which is an interesting problem for Parallels and VMWare. I would imagine the key requirement for most of their Apple customers is to run Windows. On the other hand it might be an opportunity to offer Windows in the cloud to Apple users.
p1mrx 6y ago
ಠ_ಠ when they refer to x86_64 as "Intel instructions": https://en.wikipedia.org/wiki/X86-64#AMD64 https://en.wikipedia.org/wiki/X86-64#AMD64
- gabagool 6y ago
  I don't think Apple has sold any AMD Macs, correct? I understand your consternation, though. Dev docs shouldn't require dumbing things down.
Mister_Snuggles 6y ago
I've been using Macs at work for a long time, but sadly this will mark the end of that era. In particular, this limitation on Rosetta rules out an ARM-based Mac for work: > Virtual Machine apps that virtualize x86_64 computer platforms My job requires me to use a piece of proprietary Windows-only software for a large portion of my work. If I can't use this software I can't do my job. Currently I run it in VMware Fusion on an Intel Mac, which is a perfect solution for me - I get the great features of MacOS, plus I can run the proprietary toolset that my job requires. There is a very remote possibility that Windows for ARM could be virtualized by some future version of VMware and the proprietary toolset could run under that, but I'm not holding my breath. Due to budget constraints, I don't think there's any way that my work would spring for a MacBook Pro plus a Windows machine for me. On the flip side, Windows 10 seems to be getting really good, so I expect that I'll be just as happy and productive with Windows 10+WSL2.
- empthought 6y ago
  It will be at least several years before you will not be able to purchase a new Intel-based Mac.
  - Mister_Snuggles 6y ago
    From another article, Apple claims that the transition will be complete in two years. I have 2.5 years left before I will be eligible to have my work computer replaced. Depending on the exact timing of Apple's product cycles and the procurement cycle, I'll either be getting one of the last Intel Macs or a Windows machine of some description. If an Intel Mac is still available at this time, that brings other concerns - in particular, Macs are on a five-year replacement cycle at my work, so even if they are available it may not be prudent to buy one. Will it get new MacOS versions for its 5-year lifetime? Will it still get security updates five years down the road? Will the developers of the applications I use still ship x86_64 versions of their Mac applications? Because of all of that uncertainty, it may end up being a better option to just get a Windows machine when I'm eligible. Fortunately, a lot of that uncertainty should be cleared up by the time that happens. EDIT: Another remote possibility is that VMware figures out some way to fill in the gaps that Rosetta doesn't. After all, they did figure out how to do virtualization on a platform that didn't even support it.
loph 6y ago
What's old is new again. This reminds me of Digital's VEST technology. VEST would convert VAX programs to run on Alpha. From 32-bit CISC to 64-bit RISC. Nearly 30 years ago. https://web.stanford.edu/class/cs343/resources/binary-translation.pdf https://web.stanford.edu/class/cs343/resources/binary-transl...
- reaperducer 6y ago
  Or the PET emulator from 38 years ago that Commodore released because it wasn't sure that the C-64 would have any software. Ditto for the C-64 CP/M cartridge of the same vintage.
  - classichasclass 6y ago
    The PET emulator was an interesting case. It relied on the fact that BASIC 2.0 on the C64 was similar enough to that on the PET (and IEC serial devices similar enough to IEEE-488 ones, from the BASIC side, at least) that many programs ran unmodified. The interesting bits were adding some goop so that many common POKEs and PEEKs from BASIC would "just work" as well (like CB audio, screen memory, etc.). This was even enough for some machine language programs, though many choked on the different memory map.
slaymaker1907 6y ago
I wonder how/if it supports unaligned jumps which x86 supports IIRC. The consequence of unaligned jumps is that it can effectively make it impossible to know the set of instructions a binary might use.
benmmurphy 6y ago
is the translation 100% static? for example if I do something like: function_pointer = blackbox_function_translator_does_not_understand(); function_pointer() then how do we end up calling the correct address in arm64 land? there are no type tags in assembly to distinguish between integers and addresses. in QEMU my understanding is translation is done dynamically so addresses would float around in memory as x86_64 addresses and then when you tried to `call` them it would look up a mapping table. In QEMU I suspect they also try and optimise this case similar to a JIT using inline caching so most of the time you wouldn't actually hit the mapping table. But if you are not dynamically converting x86_64 addresses to arm64 addresses then you need to understand what all the addresses in the program are and understand all the manipulations that might be performed on those addresses. now, you shouldn't actually be doing weird manipulations of addresses to functions in memory but if you are running obfuscated code this often happens. I think in QEMU this would work assuming (myfun+4)() does something intelligent: uint64_t add(uint64_t v) { return v + 4; } void myfun(); int main() { void (*x)() = &myfun; x = (void*)add((uint64_t)x); x(); printf("%lld\n", add(4)); } if you are holding function addresses as arm64 addresses in memory then you need to dispatch add() based on whether the argument is an integer or an address.

221 points by loun 6y ago

The ability to switch a universal binary between running x86 and ARM versions so as to support non-updated x86 plugins is really cool.

dsabanin 6y ago

From the page: What Can't Be Translated? Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. However, Rosetta doesn’t translate the following executables: * Kernel extensions * Virtual Machine apps that virtualize x86_64 computer platforms Some people are not going to be happy about this. Edit: But I personally am okay with that.

yjftsjthsd-h 6y ago
Like... it's not ideal, maybe, but kernel extensions are already a little touchy so I'm not surprised that they at least need to recompile, and VMs likewise are unfortunate but they can just rebuild and be good (i.e. I'm betting less than a month before people are running around using qemu-system-x86_64 natively built for ARM again).
unix_fan 6y ago
Meh, kernel extensions are on life support at this point.
perardi 6y ago
Apple wants kexts gone. They announced the deprecation last year, and this is another chance to force the change. (And for good reasons, if my layman's understanding of the security is at all correct.) [https://developer.apple.com/support/kernel-extensions/](https://developer.apple.com/support/kernel-extensions/) https://developer.apple.com/support/kernel-extensions/](http...
- selsta 6y ago
  Apple ARM will support kexts. https://developer.apple.com/documentation/apple_silicon/installing_a_custom_kernel_extension https://developer.apple.com/documentation/apple_silicon/inst...
  - perardi 6y ago
    Well color me surprised. I figured that’d be on the chopping block.
    - monocasa 6y ago
      I was surprised too. I wonder which crazy peripheral (or maybe it was antivirus/intrusion detection?) made them not go all in on "only apple gets to be in kernel space".
      - perardi 6y ago
        Judging by this horrible Bitdefender Endpoint Security that IT has foisted upon me: device management and anti-malware stuff.
      - saagarjha 6y ago
        Maybe Dropbox?

marcinzm 6y ago

>However, Rosetta doesn’t translate the following executables: Virtual Machine apps that virtualize x86_64 computer platforms How will this impact docker? Does this mean you can't run x86 docker containers on new Apple laptops?

cloogshicer 6y ago
Not sure about the details for how this works, but they specifically mentioned Docker to be supported at the keynote.
- marcinzm 6y ago
  x86 docker or just docker? There's ARM docker containers as well.
- dividedbyzero 6y ago
  Docker already runs fine on ARM64, like on Raspberry Pis, so there would be no Rosetta translation needed for ARM64 images. If Rosetta could translate binaries in x86/AMD64 images, that would be very helpful indeed. Likewise, will it support CLI programs at all?
EtienneK 6y ago
I would assume you’d only be able to run ARM Docker containers.
- marcinzm 6y ago
  Which is less than ideal if you're using x86 servers since you no longer have the same dev containers as production containers.
  - EtienneK 6y ago
    Yeah. I predict a sudden increase of ARM servers as well.

tomduncalf 6y ago

Are the limitations on x86_64 virtualization likely to be for technical reasons, or patent reasons? I read a comment on here alluding to some patents on x86_64 virtualization expiring later this year: https://news.ycombinator.com/item?id=23612256 https://news.ycombinator.com/item?id=23612256 - could that mean that there is a chance this might happen and they are keeping it quiet for now, or are patents likely unrelated?

unicornfinder 6y ago
If I had to guess it's because Rosetta isn't fully emulating an x86_64 instruction set but rather dynamically translating API calls for the most part.
- mkl 6y ago
  I think it's neither of those things. The x86_64 machine code instructions are translated into ARM instructions, so there's no emulation and the API calls don't need to change.
  - monocasa 6y ago
    > The x86_64 machine code instructions are translated into ARM instructions, so there's no emulation That's what emulation is.
    - mkl 6y ago
      I think emulation would be interpreting the instructions one by one and doing what they say in software (a bit like Python's byte code interpreter). Rosetta translates the binary to another (now native) binary, and then runs it like normal, and much of the time this will be an ahead-of-time once off translation.
      - monocasa 6y ago
        Recompilation is a valid emulation strategy. You're just amortizing client instruction decode by doing it ahead of time. Last time I checked, that's how RPCS3 worked for instance, since PS3's GameOS didn't generally allow JITs anyway.

underdeserver 6y ago

Translating JITted code is a pretty cool trick.

saagarjha 6y ago
At runtime it doesn't really matter where you code is coming from, as long as you do flushes at the right spots.
- jeffbee 6y ago
  But doesn't this mean that they are translating and running the JIT and the JIT is creating new executable x86 code which then also needs to be translated? I'm a bit baffled as to how any of this can work properly, for example if my JIT is sampling instructions to determine when to optimize, does Rosetta need to reverse-translate the actual instruction pointer back to the original x86 code offset?
  - underdeserver 6y ago
    > But doesn't this mean that they are translating and running the JIT and the JIT is creating new executable x86 code which then also needs to be translated? Yup. > I'm a bit baffled as to how any of this can work properly, for example if my JIT is sampling instructions to determine when to optimize, does Rosetta need to reverse-translate the actual instruction pointer back to the original x86 code offset? Well, not exactly. Let's take JS as an example. First of all, you're only sampling calls that are still in the native language, so your instruction pointer is in the emulated virtual machine itself anyway, which is agnostic to the CPU architecture. See [1] for where SpiderMonkey samples. What happens then is that the engine sees a function being called many times, so the JIT compiler decides to compile a function; it compiles the function and writes the compiled function to memory, then overwrites that function's address with a native function pointer. The easiest hook the OS has to this process is mprotect(2). In modern OSes, memory is generally either writable or executable, but not both; so if you want to compile and write, and then execute, you need to call mprotect() between those steps to set permissions. Knowing that your binary is Rosetta'd, MacOS can assume your JIT code is also AMD64 and translate that page on the fly. All they have to do is make sure the virtual memory addresses are correct - to not mess up those function pointers. They can either do it lazily, through a pagefault handler, or greedily when mprotect is called to add executable permissions to the JIT-allocated memory. [1] https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/JIT_Optimization_Strategies https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Sp...

nickcw 6y ago

I thought this was the most interesting paragraph. > What Can't Be Translated? > Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. I guess translation of JIT compiled stuff implies this isn't a once of translation. I guess translating plugins implies that too. It sounds like very clever stuff to me! > However, Rosetta doesn’t translate the following executables: > > Kernel extensions Fair enough > Virtual Machine apps that virtualize x86_64 computer platforms I guess most VMs rely on hardware virtualization which would be tricky to translate well. > Rosetta translates all x86_64 instructions, but it doesn’t support the execution of some newer instruction sets and processor features, such as AVX, AVX2, and AVX512 vector instructions. If you include these newer instructions in your code, execute them only after verifying that they are available. For example, to determine if AVX512 vector instructions are available, use the sysctlbyname function to check the hw.optional.avx512f attribute. These sound like they should be relatively straight forward to translate. I wonder why they didn't? Lack of time? Or perhaps because translating them means that they don't run fast enough to be useful and the fallback paths are likely to run quicker.

leeter 6y ago
RE AVX and above: My guess is that they are licensing the Transmeta IP which doesn't include those licenses IIRC. So unless they wanted to get sued they'd have to be careful. I suspect some SSE instructions may also not be supported as I don't recall if Transmeta had a license up to SSE4.
- monocasa 6y ago
  I doubt they're licencing Transmeta. Transmeta never had an x86_64 processor.
  - leeter 6y ago
    True, but they did have the license IIRC. Otherwise they'd be licensing from VIA or whomever owns that license now. That license AFAIK would include up to SSE4. It's not like Apple is going to tell us. But the JIT nature makes me think it's Transmeta as the company currently holding that IP is a "Patent enforcement entity".

jayyhu 6y ago

>Rosetta translates all x86_64 instructions, but it doesn’t support the execution of some newer instruction sets and processor features, such as AVX, AVX2, and AVX512 vector instructions. No AVX will probably mean that the vast majority of pro/graphics intensive apps won't work out of the box with Rosetta.

beervirus 6y ago
Do those programs blindly try to execute AVX instructions without checking whether they’re supported? What happens if you're just running on an older intel processor?
- saagarjha 6y ago
  Usually you'd want to check cpuid or similar and fall back to something reasonable.
- jayyhu 6y ago
  Apple seems to suggest that all well behaving programs: >use the sysctlbyname function to check the hw.optional.avx512f attribute So my guess it they probably just replace the AVX instructions with no-ops and let it blow up if the correct checks/fallbacks aren't in place.
  - monocasa 6y ago
    Or just SIGILL rather than noop.
  - throwaway2048 6y ago
    it would be a lot more normal to trap the illegal instruction, the same thing that happens if you give a CPU a nonsense nonexistent instruction.
- makomk 6y ago
  All well-behaving software will of course check if AVX is supported before trying to use it, but at least in some cases that just means they'll refuse to install or run on systems that don't have it: https://www.pro-tools-expert.com/production-expert-1/2019/9/5/is-this-software-issue-the-final-curtain-call-for-old-mac-pro-cheese-graters-used-in-todays-recording-studios https://www.pro-tools-expert.com/production-expert-1/2019/9/... (Developing parallel code paths and keeping them working indefinitely is a lot of extra work, and AVX has been around for close to a decade at this point.)

bjpirt 6y ago

Having big flashbacks to the switch from PPC to x86 here. Rosetta worked relatively smoothly during that transition so fingers crossed it will be ok here too. Though with Docker support on the mac already being a second class citizen to running on Linux I wonder if a lot of devs will stop using macs for dev

LeSaucy 6y ago
Highly unlikely Apple would cede the software engineering market for a competitor to step in. Almost everyone in my company would prefer to have a MacBook vs a Thinkpad/Dell/etc if given the choice.
- SomeHacker44 6y ago
  Speaking solely for myself, I do not intend to go with this experiment after using Macs for 16 years and change. I have found that with a few rough edges, Windows 10 and WSL fulfills my dev needs, and except for the Apple apps Logic and Final Cut Pro, virtually all my software is cross platform. Besides, I can use Ableton and Adobe Premiere which I also already "own."
  - jjcon 6y ago
    > I do not intend to go with this experiment I’m sure they lost people when they switched off of ppc too but just like last time Apple is playing the long game. Intel just can’t keep up anymore.
    - PaulDavisThe1st 6y ago
      Intel can't (maybe). AMD can. In fact right now, AMD doesn't even have to keep up, because it's substantially out in front.
      - jjcon 6y ago
        AMD has worse thermals than intel does which is likely the reason Apple is switching in the first place

godzillabrennus 6y ago

The Apple transition to ARM for me is sad. There was so much positivity around the Intel transition. It opened up the Mac platform. Now it’s going back into a closed black box.

Bigpet 6y ago
I think I'm missing something. How is ARM closing things up? Does Apple have a bunch of processor extensions that make it incompatible with other ARM processors? Or do you think they'll take the chance to kill boot-camp or something?
- monocasa 6y ago
  They'll almost certainly only allow their signed kernels to run. And that's in addition to not documenting any of the peripherals.
  - larkost 6y ago
    Apple already has a program for select third party companies to get their own signing keys for kernel extensions. They have been tightening that circle over the last few years as they provide more and more generic interfaces so that companies can replace KEXTs with user space code. From what they said in the State of the Union keynote they want to get that down to nothing, but realize they can't do that yet. No details were provided, but there might be more in the KEXT sessions if you are interested.
- proninyaroslav 6y ago
  I think Apple has more maneuvers on ARM, since in general the PC architecture is open for modifications, unlike ARM. They will do it the same way as with iOS devices - a closed bootloader and hardware protection.
- qubex 6y ago
  The “ARM world” is almost inconceivably varied by x86 standards. There’s not even an agreed-upon boot procedure (the Raspberrry Pi’s CPU famously gets booted by the GPU). Efforts to standardise this in the server space are only just getting off the ground with the Arm Server Base Boot Requirements version 1.2 being ratified last year. I doubt you’ll find Apple subscribing to anything that they won’t develop in-house. This is, after all, the company that developed the Lightening connector and sticks on its iPhones to it despite the whole industry shifting to USB-C. Basically the OS situation on Macs post-ARM is probably going to look like the Linux situation on iDevices currently: occasionally somebody manages to text-mode-boot an iPhone 7 into a profoundly unstable hacked-up kernel and there is much merriment and rejoicing, but really... macs are going to become rather exotic (on the inside). Have you ever wondered about your iPad’s boot sequence? Because you’ve never had to think about it. Notice how you can’t install versions of i(Pad)OS that are no longer signed? You think Apple will allow booting unsigned images? And all the Secure Enclave infrastructure they’ve built up? Nah.

DCKing 6y ago

Rosetta's goal is to support legacy Mac apps. It's quite finely scoped, but I hope Apple will go above this and make it available as part of their virtualization framework. This way e.g. Parallels and Docker could use this to provide some way to tap in probably the fastest way to run x86 on ARM Macs. This would mean Rosetta would go beyond its stated scope, and certainly go beyond what the relatively short-lived Rosetta 1 did, but would make it easier to integrate ARM Macs in what is still an x86 world. I'm worried they're not going to do this because x86 emulation is probably too slow (people will attempt to run their AAA games in a VM). It would also mean that Apple will need to support Rosetta 2 forever. If they're not going to do this, everybody will have to rely on qemu. Qemu is great, but I hope it will perform adequately for at least basic Docker stuff...

Angostura 6y ago
FWIW they showed the latest Tombraider game running on Rosetta on a dev Mac said to be using the ARM chip from the high-end iPad. It ran pretty well. Not sure if I would quite count this as AAA
- DCKing 6y ago
  I don't think that was running on Rosetta 2's dynamic recompiler, but instead used Rosetta 2's other stated feature of being able to translate existing apps at install time. In the demo they state that they downloaded the game from the Mac App store. Certainly static recompilation yields somewhat-close-to-native performance when it is available, but that is not possible for x86 "VMs".
- toyg 6y ago
  > It ran pretty well It was a postproduced, recorded promotional video, not a live demo. They could have shown laptops reaching Mars and you’d have thought “that landing gear looks pretty solid”.
- monocasa 6y ago
  IDK, what I saw was a several year old game running on the lowest settings and still stuttering a bit.

mkl 6y ago

> Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. How on Earth does it do that? If executable code is being generated at runtime, it's going to be x86_64 binary machine code still (there are too many ways to generate valid machine code, and it won't know right away whether you're JITting, or cross compiling and actually want x86_64), so Rosetta would need to detect when the code's about to be run, or when it's marked as executable, and translate the machine code in that bit of memory just in time. The length of the ARM code might be longer, so it would have to be in a different block of memory, with the original x86_64 code replaced with a jump to the new code or something. It's late at night here, so maybe I'm missing a simpler approach, but I'm a bit surprised they have it working reliably enough to make such a general statement (there being a great variety of JIT systems). From a quick search I can't tell if Microsoft's x86-on-ARM translation in Windows 10 ARM supports JITs in the program being run.

lsllc 6y ago
It probably just hooks mmap() (or mprotect()) and looks for when the JIT sets PROT_EXEC on memory it hasn't translated yet. Here's a tutorial for a simple JIT that uses mmap() with PROT_EXEC to write machine code to before executing: https://github.com/spencertipping/jit-tutorial https://github.com/spencertipping/jit-tutorial
- monocasa 6y ago
  Since they don't enforce W^X you need to go a little deeper than that since some JITs reuse the same memory later for new traces. So what you'll do is silently enforce W^X to trap code modifications, but not propagate those traps to the emulated code.
  - lsllc 6y ago
    Right, if you look at the example JIT project I posted, they start by creating the memory with mmap() and PROT_EXEC before generating any code. So yes, you'd need to trap subsequent writes to allow for retranslation [I assume they have hooks in the Darwin kernel for this].

nothis 6y ago

So how fast will this be compared to native? As I understand, it's not "emulation" but something more low level?

Synaesthesia 6y ago
Did you ever use PPC apps on Intel through Rosetta? It was a substantial slowdown, like at least 2x. But the app was usable.

rz2k 6y ago

Does anyone have any insight into the business logistics involved in a transition like this? I presume Apple has done maintenance on transitioning desktop OS X to ARM as an option for a very long time. How many years ago would they have had to decide that was the direction they were going to make it reality? 2015? How many people would have been working on it? How many billions of dollars? How does the cost of developing the software compare to the cost of developing a new processor, or compared to tooling a production line for an entirely new platform? I'm really curious about the share of institutional resources and attention this likely involved. I wonder how important it is as context to Apple over the past few years. I also wonder if it heralds a new direction in personal computers. Every time I've considered that my Mac isn't that great, the alternatives aren't that great either. Would Apple ever decide to seriously compete for a dominant share of the personal computer market? And finally, I am also curious about the accounting and financial disclosures required in such decisions. How much are institutional investors informed of long range plans? There's naturally a lot more pressure to distribute cash to shareholders if you don't know about some five-year-plan, yet that pressure waxes and wanes from activist investors, and institutional investors like CalPERS always seemed to be on board with Apple's direction anyway. Do unofficial leaks prevent violations of insider trading rules?

ryanianian 6y ago
> How many billions of dollars? Rounding-error-sized marginal costs. Even if you don't plan on pivoting architectures, it's good practice to write as much code as possible without assembly/intrinsics. If you're writing standards-compliant C/C++/Objective-C/Swift/* it's not a huge burden to change architectures. They also have past-experience with such transitions, so hopefully they kept the "keep the code portable" mantra close over the past 10+ years. The choice to do this seems relatively obvious for Apple. They have spent years building up their silicon game with portables. Their frustration with Intel is also palpable, so it was probably "just" a waiting game (not "how" or "if") in deciding when to pull the trigger and start using those chips outside of iPhones/iPads.
- Someone 6y ago
  If you have to maintain any highly optimized assembly/intrinsics code, it’s a very good idea to also maintain an “as portable as feasible” C version for comparison testing.

andoma 6y ago

I wonder how it handles the stricter memory ordering from x86? For example, this code: void store_value_and_unlock(int *p, int *l, int v) { *p = v; __sync_lock_release(l, 0); } on x86_64 compiles to: mov dword ptr [rdi], edx mov dword ptr [rsi], 0 ret vs on arm64: str w2, [x0] stlr wzr, [x1] ret Notice how the second store on ARM is a store with release semantics to ensure correct memory ordering as it was intended in the original C code. This information is lost as it's not needed on x86 which guarantees (among other things) that stores are visible in-order from all other threads.

monocasa 6y ago
That's the big piece I've been wondering about too. Three options as I see it (none of them great): 1) Pin all threads in an x86 process to a single core. You don't have memory model concerns on a single core. 2) Don't do anything? Just rely on apps to use the system provided mutex libraries, and they just break if they try to roll their own concurrency? Seems like exactly the applications you care about (games, pro apps), would be the ones most likely to break. 3) Some stricter memory model in hardware? Seems like that'd go against most of the stated reason for switching to ARM in the first place.
- andoma 6y ago
  > 3) Some stricter memory model in hardware? Seems like that'd go against most of the stated reason for switching to ARM in the first place. I would assume that a more strict memory model would be enabled only for processes that needs it (ie, Rosetta translated ones). So a cpu-flag is set/cleared when entering/exiting user mode for those processes. Does this require a separate/special cache coherency protocol? A complete L1d flush when entering/leaving these processes (across all CPUs)? Not and expert in this field and it feels complicated for sure. Is it worth it for just emulating "legacy" applications during a transitional period? Perhaps Apple can pull it off though.
  - monocasa 6y ago
    > I would assume that a more strict memory model would be enabled only for processes that needs it (ie, Rosetta translated ones). So a cpu-flag is set/cleared when entering/exiting user mode for those processes. Does this require a separate/special cache coherency protocol? The benefits you'd get from going to a weaker memory model are by not having that extra coherency in the critical path in the first place. Adding extra muxes in front of it to make it optional would be worse than just having it on all the time. > A complete L1d flush when entering/leaving these processes (across all CPUs)? That wouldn't help because two threads could be running at the same time on different cores against their respective L1 and store buffers.

viro 6y ago

So does this mean no windows VMs?

tonyedgecombe 6y ago
Yes, which is an interesting problem for Parallels and VMWare. I would imagine the key requirement for most of their Apple customers is to run Windows. On the other hand it might be an opportunity to offer Windows in the cloud to Apple users.

p1mrx 6y ago

ಠ_ಠ when they refer to x86_64 as "Intel instructions": https://en.wikipedia.org/wiki/X86-64#AMD64 https://en.wikipedia.org/wiki/X86-64#AMD64

gabagool 6y ago
I don't think Apple has sold any AMD Macs, correct? I understand your consternation, though. Dev docs shouldn't require dumbing things down.

Mister_Snuggles 6y ago

I've been using Macs at work for a long time, but sadly this will mark the end of that era. In particular, this limitation on Rosetta rules out an ARM-based Mac for work: > Virtual Machine apps that virtualize x86_64 computer platforms My job requires me to use a piece of proprietary Windows-only software for a large portion of my work. If I can't use this software I can't do my job. Currently I run it in VMware Fusion on an Intel Mac, which is a perfect solution for me - I get the great features of MacOS, plus I can run the proprietary toolset that my job requires. There is a very remote possibility that Windows for ARM could be virtualized by some future version of VMware and the proprietary toolset could run under that, but I'm not holding my breath. Due to budget constraints, I don't think there's any way that my work would spring for a MacBook Pro plus a Windows machine for me. On the flip side, Windows 10 seems to be getting really good, so I expect that I'll be just as happy and productive with Windows 10+WSL2.

empthought 6y ago
It will be at least several years before you will not be able to purchase a new Intel-based Mac.
- Mister_Snuggles 6y ago
  From another article, Apple claims that the transition will be complete in two years. I have 2.5 years left before I will be eligible to have my work computer replaced. Depending on the exact timing of Apple's product cycles and the procurement cycle, I'll either be getting one of the last Intel Macs or a Windows machine of some description. If an Intel Mac is still available at this time, that brings other concerns - in particular, Macs are on a five-year replacement cycle at my work, so even if they are available it may not be prudent to buy one. Will it get new MacOS versions for its 5-year lifetime? Will it still get security updates five years down the road? Will the developers of the applications I use still ship x86_64 versions of their Mac applications? Because of all of that uncertainty, it may end up being a better option to just get a Windows machine when I'm eligible. Fortunately, a lot of that uncertainty should be cleared up by the time that happens. EDIT: Another remote possibility is that VMware figures out some way to fill in the gaps that Rosetta doesn't. After all, they did figure out how to do virtualization on a platform that didn't even support it.

loph 6y ago

What's old is new again. This reminds me of Digital's VEST technology. VEST would convert VAX programs to run on Alpha. From 32-bit CISC to 64-bit RISC. Nearly 30 years ago. https://web.stanford.edu/class/cs343/resources/binary-translation.pdf https://web.stanford.edu/class/cs343/resources/binary-transl...

reaperducer 6y ago
Or the PET emulator from 38 years ago that Commodore released because it wasn't sure that the C-64 would have any software. Ditto for the C-64 CP/M cartridge of the same vintage.
- classichasclass 6y ago
  The PET emulator was an interesting case. It relied on the fact that BASIC 2.0 on the C64 was similar enough to that on the PET (and IEC serial devices similar enough to IEEE-488 ones, from the BASIC side, at least) that many programs ran unmodified. The interesting bits were adding some goop so that many common POKEs and PEEKs from BASIC would "just work" as well (like CB audio, screen memory, etc.). This was even enough for some machine language programs, though many choked on the different memory map.

slaymaker1907 6y ago

I wonder how/if it supports unaligned jumps which x86 supports IIRC. The consequence of unaligned jumps is that it can effectively make it impossible to know the set of instructions a binary might use.

benmmurphy 6y ago

is the translation 100% static? for example if I do something like: function_pointer = blackbox_function_translator_does_not_understand(); function_pointer() then how do we end up calling the correct address in arm64 land? there are no type tags in assembly to distinguish between integers and addresses. in QEMU my understanding is translation is done dynamically so addresses would float around in memory as x86_64 addresses and then when you tried to `call` them it would look up a mapping table. In QEMU I suspect they also try and optimise this case similar to a JIT using inline caching so most of the time you wouldn't actually hit the mapping table. But if you are not dynamically converting x86_64 addresses to arm64 addresses then you need to understand what all the addresses in the program are and understand all the manipulations that might be performed on those addresses. now, you shouldn't actually be doing weird manipulations of addresses to functions in memory but if you are running obfuscated code this often happens. I think in QEMU this would work assuming (myfun+4)() does something intelligent: uint64_t add(uint64_t v) { return v + 4; } void myfun(); int main() { void (*x)() = &myfun; x = (void*)add((uint64_t)x); x(); printf("%lld\n", add(4)); } if you are holding function addresses as arm64 addresses in memory then you need to dispatch add() based on whether the argument is an integer or an address.