22 ms·

Why is Rosetta 2 fast?

650 points by pantalaimon 4y ago

spullara 4y ago
The first time I ran into this technology was in the early 90s on the DEC Alpha. They had a tool called "MX" that would translate MIPS Ultrix binaries to Alpha on DEC Unix: https://www.linuxjournal.com/article/1044 https://www.linuxjournal.com/article/1044 Crazy stuff. Rosetta 2 is insanely good. Runs FPS video games even.
deleted 4y ago
[deleted]
0x000xca0xfe 4y ago
I wonder if such a direct translation from ARM to another architecture would even be possible given that the instruction set can be changed at runtime (thumb mode). Does anybody know how often typical ARM32 programs execute this mode switching or if such sections can be recognized statically?
- anyfoo 4y ago
  If that's really an issue (most code probably sticks to one mode, but I have no data), just translate twice, once in either mode. ¯\_(ツ)_/¯
- saagarjha 4y ago
  Shouldn’t be hard under this scheme, it tracks indirect branches anyways. Just swap which table you do lookups based on whether you’re in thumb mode or not.
- astrange 4y ago
  x86 is worse because you can jump into the middle of instructions. In that case you just fall back to JIT though. And luckily AArch64 doesn't have Thumb.
musicale 4y ago
Rosetta 2 is beautiful - I would love it if they kept it as a feature for the long term rather than deprecating it and removing it in the next release of macOS (basically what they did during previous architectural transitions.) If Apple does drop it, maybe they could open source it so it could live on in Linux and BSD at least. ;-) Adding a couple of features to ARM to drastically improve translated x86 code execution sounds like a decent idea - and one that could potentially enable better x86 app performance on ARM Windows as well. I don't know the silicon cost but I'd hope it wasn't dropped in the future.' Thinking a bit larger, I'd also like to see Apple add something like CHERI support to Apple Silicon and macOS to enable efficient memory error checking in hardware. I'd be surprised if they weren't working on something like this already.
webwielder2 4y ago
(Apologies for the flame war quality to this comment, I’m genuinely just expressing an observation) It’s ironic that Apple is often backhandedly complimented by hackers as having “good hardware” when their list of software accomplishments is amongst the most impressive in the industry and contrasts sharply with the best efforts of, say, Microsoft, purportedly a “software company.”
- majou 4y ago
  The transparent APFS migration was one heck of a feat.
- solarkraft 4y ago
  Both have pretty impressive engineering. The focus just appears to be different. Apple really cares about user experience (or, maybe, used to), while it seems like Microsoft is trying to (and according to what I hear mostly succeeding) nail the administration experience instead.
fooblaster 4y ago
A lot of the simplicity of this approach relies on the x86 registers being able to be directly mapped to arm registers. This seems to be possible for most x86 registers, even simd registers. Although, I think this falls over for avx512, which is supported on the Mac pro. Arm neon has 32 128 bit registers - avx512 supports 32 512 bit registers and dedicated predicate registers. What do they do? Back to jit mode?
- fooblaster 4y ago
  oh, they don't support avx: https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment https://developer.apple.com/documentation/apple-silicon/abou...
fulafel 4y ago
For history this was a major milestone in x86 binary translation, Digital FX!32: https://www.usenix.org/legacy/publications/library/proceedings/usenix-nt97/full_papers/chernoff/chernoff.pdf https://www.usenix.org/legacy/publications/library/proceedin... Some apps run faster than the fastest available x86 at the time, and sometimes significantly faster like the Byte benchmark cited above. Of course it helped that the chip was sugnificantly faster than leading x86 chips in the first place.
johnthuss 4y ago
"I believe there’s significant room for performance improvement in Rosetta 2... However, this would come at the cost of significantly increased complexity... Engineering is about making the right tradeoffs, and I’d say Rosetta 2 has done exactly that."
- Gigachad 4y ago
  Would be a waste of effort when the tool is designed to be obsolete in a few years as everything gets natively compiled.
  - solarkraft 4y ago
    They could've amazed a few people a bit more by emulating x86 apps even faster (but M1+Rosetta can already run some stuff faster than an Intel Mac), but then the benefit of releasing native apps would be much decreased ("why bother, it's good enough ..."). It's a delicate political game that they, yet again, seem to be playing pretty well.
MikusR 4y ago
The main reason, M1/2 being incredibly fast. Is listed last.
- dagmx 4y ago
  Perhaps if you’re comparing against Intel processors, but even on an Apple Silicon Mac, Rosetta 2 vs native versions of apps are no slouch. 20% overhead for a non-native executable is very commendable.
- postalrat 4y ago
  That isn't the main reason. If Rosetta ran x86 code at 10% the speed of native nobody would be calling it fast.
- Someone 4y ago
  I don’t think that’s the main reason. The article lists a few things that, I think the main reason is that they made several parts of the CPU behave identical to x86. The M1 and M2 chips: - can be told to do total store ordering, just as x86 does - have of a few status flags that x86 has, but regular arm doesn’t - can be told to make the FPU behave exactly as the x86 FPU It also helps that ARM has many more registers than x86. Because of that the emulator can map the x86 registers to ARM registers, and have registers to spare for use by the emulator.
  - astrange 4y ago
    None of those custom features are used in Linux/in VMs, and it's still fairly fast.
Vt71fcAqt7 4y ago
I hope Rosetta is here to stay and continues developement. And I hope what is learned from it can be used to make a RISC-V version of it. translating native ARM to RISC-V should be much easier than x86 to ARM as I understand it, so one could conceivably do x86 -> ARM -> RISC-V.
- masklinn 4y ago
  > I hope Rosetta is here to stay and continues developement. It almost certainly is not. Odds are Apple will eventually remove Rosetta II, as they did Rosetta back in the days, once they consider the need for that bridge to be over (Rosetta was added in 2006 in 10.4, and removed in 2011 from 10.7). > And I hope what is learned from it can be used to make a RISC-V version of it. translating native ARM to RISC-V should be much easier than x86 to ARM as I understand it, so one could conceivably do x86 -> ARM -> RISC-V. That's not going to happen unless Apple decides to switch from ARM to RISC-V, and... why would they? They've got 15 years experience and essentially full control on ARM.
  - quux 4y ago
    It would be funny/not funny if in a few years Apple removes Rosetta 2 for Mac apps but keeps the Linux version forever so docker can run at reasonable speeds.
  - stu2b50 4y ago
    Rosetta 1 had a ticking time bomb. Apple was licensing it from a 3rd party. Rosetta 2 is all in house as far as we know. Different CEO as well. Jobs was more opinionated on “principles” - Cook is more than happy to sell what people will buy. I think Rosetta 2 will last.
    - deleted 4y ago
      [deleted]
    - TillE 4y ago
      What important Intel-only macOS software is going to exist in five years? It's basically only games and weird tiny niches, and Apple is pretty happy to abandon both those categories. The saving grace is that there's very few interesting Mac-exclusive games in the Intel era.
hinkley 4y ago
Apple is doing some really interesting but really quiet work in the area of VMs. I feel like we don’t give them enough credit but maybe they’ve put themselves in that position by not bragging enough about what they do. As a somewhat related aside, I have been watching Bun (low startup time Node-like on top of Safari’s JavaScript engine) with enough interest that I started trying to fix a bug, which is somewhat unusual for me. I mostly contribute small fixes to tools I use at work. I can’t quite grok Zig code yet so I got stuck fairly quickly. The “bug” turned out to be default behavior in a Zig stdlib, rather than in JavaScript code. The rest is fairly tangential but suffice it to say I prefer self hosted languages but this probably falls into the startup speed compromise. Being low startup overhead makes their VM interesting, but the fact that it benchmarks better than Firefox a lot of the time and occasionally faster than v8 is quite a bit of quiet competence.
- jraph 4y ago
  > feel like we don’t give them enough credit but maybe they’ve put themselves in that position by not bragging enough about what they do. And maybe also by keeping the technology closed and Apple-specific. Many people who could be interested in using it don't have access to it.
  - jolux 4y ago
    WebKit B3 is open source: https://webkit.org/docs/b3/ https://webkit.org/docs/b3/
    - deleted 4y ago
      [deleted]
  - freedomben 4y ago
    Exactly. As someone who would be very interested in this, but don't use Apple products, it's just not exciting because it's not accessible to me (I can't even test it as a user). If they wanted to write a whitepaper about it to share knowledge, that might be interesting, but given that it's Apple I'm not gonna hold my breath.
peatmoss 4y ago
Not having any particular domain experience here, I've idly wondered whether or not there's any role for neural net models in translating code for other architectures. We have giant corpuses of source code, compiled x86_64 binaries, and compiled arm64 binaries. I assume the compiled binaries represent approximately our best compiler technology. It seems predicting an arm binary from an x86_64 binary would not be insane? If someone who actually knows anything here wants to disabuse me of my showerthoughts, I'd appreciate being able to put the idea out of my head :-)
- qsort 4y ago
  You would need a hybrid architecture with a NN generating guesses and a "watchdog" shutting down errors. Neural models are basically universal approximators. Machine code needs to be obscenely precise to work. Unless you're doing something else in the backend, it's just a turbo SIGILL generator.
  - throw10920 4y ago
    This is all true - machine code needs to be "basically perfect" to work. However, there are lots of problems in CS that are easier to check the answer to a solution than to solve in the first place. It may turn out to be the case that a well-tuned model can quickly produce solutions to some code-generation problems, that those solutions have a high enough likelihood of being correct, that it's fast enough to check (and maybe try again), and that this entire process is faster than state-of-the-art classical algorithms. However, if that were the case, I might also expect us to be able to extract better algorithms from the model - intuitively, machine code generation "feels" like something that's just better implemented through classical algorithms. Have you met a human that can do register allocation faster than LLVM?
  - classichasclass 4y ago
    > turbo SIGILL generator This gave me the delightful mental image of a CPU smashing headlong into a brick wall, reversing itself, and doing it again. Which is pretty much what this would do.
- 4y ago
menaerus 4y ago
> Rosetta 2 translates the entire text segment of the binary from x86 to ARM up-front. Do I understand correctly that the Rosetta is basically a transpiler from x86-64 machine code to ARM machine code which is run prior to the binary execution? If so, does it affect the application startup times?
- nilsb 4y ago
  Yes, it does. The delay of the first start of an app is quite noticeable. But the transpiled binary is apparently cached somewhere.
  - saagarjha 4y ago
    /var/db/oah.
- nicoburns 4y ago
  > If so, does it affect the application startup times? It does, but only the very first time you run the application. The result of the transpilation is cached so it doesn't have to be computed again until the app is updated.
  - dylan604 4y ago
    Does that essentially mean each non-native app is doubled in disk use? Maybe not doubled but requires more space to be sure.
    - saagarjha 4y ago
      Yes.
    - varenc 4y ago
      Yes... you can see the cache in /var/db/oah/ Though only the actual binary size that gets doubled. For large apps it’s usually not the binary that’s taking up most of the space.
  - 4y ago
hinkley 4y ago
I remember years ago when Java adjacent research was all the rage, HP had a problem that was “Rosetta lite” if you will. They had a need to run old binaries on new hardware that wasn’t exactly backward compatible. They made a transpiler that worked on binaries. It might have even been a JIT but that part of the memory is fuzzy. What made it interesting here was that as a sanity check they made an A->A mode where they took in one architecture and spit out machine code for the same architecture. The output was faster than the input. Meaning that even native code has some room for improvement with JIT technology. I have been wishing for years that we were in a better place with regard to compilers and NP complete problems where the compilers had a fast mode for code-build-test cycles and a very slow incremental mode for official builds. I recall someone telling me the only thing they liked about the Rational IDE (C and C++?) was that it cached precompiled headers, one of the Amdahl’s Law areas for compilers. If you changed a header, you paid the recompilation cost and everyone else got a copy. I love whenever the person that cares about something gets to pay the consequence instead of externalizing it on others. And having some CI machines or CPUs that just sit around chewing on Hard Problems all day for that last 10% seems to be to be a really good use case in a world that’s seeing 16 core consumer hardware. Also caching hints from previous runs is a good thing.
- fuckstick 4y ago
  > The output was faster than the input. So if you ran the input back through the output multiple times then that means you could eventually get the runtime down to 0.
  - deleted 4y ago
    [deleted]
  - avidiax 4y ago
    Probably the output of the decade-old compiler that produced the original binary had no optimizations.
    - hinkley 4y ago
      That too but the eternal riddle of optimizer passes is which ones reveal structure and which obscure it. Do I loop unroll or strength reduce first? If there are heuristics about max complexity for unrolling or inlining then it might be “both”. And then there’s processor family versus this exact model.
dynjo 4y ago
It is quite astonishing how seamless Apple has managed to make the Intel to ARM transition, there are some seriously smart minds behind Rosetta. I honestly don't think I had a single software issue during the transition!
- perardi 4y ago
  I think the end of support for 32-bit applications in 2019 helped, slightly, with the run-up. Assuming you weren’t already shipping 64-bit applications…which would be weird…updating the application probably required getting everything into a contemporary version of Xcode, cleaning out the cruft, and getting it compiling nice and cleanly. After that, the ARM transition was kind of a “it just works” scenario. Now, I’m sure Adobe and other high-performance application developers had to do some architecture-specific tweaks, but, gotta think Apple clued them in ahead of time as to what was coming.
- wombat-man 4y ago
  There's an annoying dwarf fortress bug but other than that, same
- ChuckNorris89 4y ago
  If that blows your mind, you should see how Microsoft did the emulation of the PowerPC based Xeon chip to X86 so you can play Xbox 360 games on Xbox One. There's an old pdf from Microsoft researchers with the details but I can't seem to find it right now.
  - dev_tty01 4y ago
    They also bought a boat load of PPC Macs for much of the early Xbox dev work.
  - RedShift1 4y ago
    Any good videos on that?
- poulpy123 4y ago
  having total control on the hardware and the software didn't hurt for sure
iainmerrick 4y ago
This is a great writeup. What a clever design! I remember Apple had a totally different but equally clever solution back in the days of the 68K-to-PowerPC migration. The 68K had 16-bit instruction words, usually with some 16-bit arguments. The emulator’s core loop would read the next instruction and branch directly into a big block of 64K x 8 bytes of PPC code. So each 68K instruction got 2 dedicated PPC instructions, typically one to set up a register and one to branch to common code. What that solution and Rosetta 2 have in common is that they’re super pragmatic - fast to start up, with fairly regular and predictable performance across most workloads, even if the theoretical peak speed is much lower than a cutting-edge JIT. Anyone know how they implemented PPC-to-x86 translation?
- klelatti 4y ago
  That’s really interesting. You might enjoy reading about the VM embedded into the Busicom calculator that used the Intel 4004 [1] They squeezed a virtual machine with 88 instructions into less than 1k of memory! [1] https://thechipletter.substack.com/p/bytecode-and-the-busicom https://thechipletter.substack.com/p/bytecode-and-the-busico...
  - hoosieree 4y ago
    And here I was feeling impressed with myself for implementing the Nand2Tetris VM translator in ~2k of python... wow. Respect for the elders!
  - iainmerrick 4y ago
    That is nifty! Sounds very similar to a Forth interpreter.
    - deleted 4y ago
      [deleted]
    - DonHopkins 4y ago
      There's also OpenFirmware's platform independent Forth bytecode "FCode": https://en.wikipedia.org/wiki/Open_Firmware https://en.wikipedia.org/wiki/Open_Firmware >Open Firmware Forth Code may be compiled into FCode, a bytecode which is independent of instruction set architecture. A PCI card may include a program, compiled to FCode, which runs on any Open Firmware system. In this way, it can provide boot-time diagnostics, configuration code, and device drivers. FCode is also very compact, so that a disk driver may require only one or two kilobytes. Therefore, many of the same I/O cards can be used on Sun systems and Macintoshes that used Open Firmware. FCode implements ANS Forth and a subset of the Open Firmware library.
retskrad 4y ago
Apple Silicon will be Tim Cook's legacy.
jeffbee 4y ago
I wonder how much hand-tuning there is in Rosetta 2 for known, critical routines. One of the tricks Transmeta used to get reasonable performance on their very slow Crusoe CPU was to recognize critical Windows functions and replace them with a library of hand-optimized native routines. Of course that's a little different because Rosetta 2 is targeting an architecture that is generally speaking at least as fast as the x86 architecture it is trying to emulate, and that's been true for most cross-architecture translators historically like DEC's VEST that ran VAX code on Alpha, but Transmeta CMS was trying to target a CPU that was slower.
- saagarjha 4y ago
  Haven’t spotted any in particular.
  - dougall 4y ago
    Yeah, I haven't either, but I haven't looked. So, I'm not sure, but I wouldn't expect any "function recognition" tricks, since there isn't really static linking, but I would expect the e.g. memcpy and strcpy implementations in the ahead-of-time translated shared cache to be written in arm64 assembly rather than translated.
    - saagarjha 4y ago
      Indeed, see the _platform_*$VARIANT$Rosetta implementations in libsystem_platform.dylib.
pjmlp 4y ago
Back in the early days of Windows NT everywhere, the Alpha version had a similar JIT emulation.
agentcooper 4y ago
I am interested in this domain, but lacking knowledge to fully understand the post. Any recommendations on good books/courses/tutorials related to low level programming?
- saagarjha 4y ago
  I’d recommend going through a compilers curriculum, then reading up on past binary translation efforts.
pjmlp 4y ago
Back in the early days of Windows NT everywhere, the Alpha version had a similar JIT emulation. https://en.m.wikipedia.org/wiki/FX!32 https://en.m.wikipedia.org/wiki/FX!32 Or for a more technical deep dive, https://www.usenix.org/publications/library/proceedings/usenix-nt97/full_papers/chernoff/chernoff.pdf https://www.usenix.org/publications/library/proceedings/usen...
- mosburger 4y ago
  OMG I forgot about FX!32. My first co-op was as a QA tester for the DEC Multia, which they moved from the Alpha processor to Intel midway through. I did a skunkworks project for the dev team attempting to run the newer versions of Multia's software (then Intel-based) on older Alpha Multias using FX!32. IIRC it was still internal use only/beta, but it worked quite well!
manv1 4y ago
Apple's historically been pretty good at making this stuff. Their first 68k -> PPC emulator (Davidian's) was so good that for some things the PPC Mac was the fastest 68k mac you could buy. The next-gen DR emulator (and SpeedDoubler etc) made things even faster. I suspect the ppc->x86 stuff was slower because x86 just doesn't have the registers. There's only so much you can do.
- Wowfunhappy 4y ago
  > Apple's historically been pretty good at making this stuff. Their first 68k -> PPC emulator (Davidian's) was so good that for some things the PPC Mac was the fastest 68k mac you could buy. Not arguing the facts here, but I'm curious—are these successes related? And if so, how has Apple done that? I would imagine that very few of the engineers who programmed Apple's 68k emulator are still working at Apple today. So, why is Apple still so good at this? Strong internal documentation? Conducive management practices? Or were they just lucky both times?
  - manv1 4y ago
    I suspect that there's a design document somewhere that has secrets and tips. Plus MacOS and NeXTstep have been ported so many times that by now any arch-level things are well-isolated. A lot of the pain of MacOS on x86 were related to kernel extensions and driver API changes. And a lot of stuff fell out during the 64-bit transition. In general, MacOS/iOS/iPadOS owe everything to the excellent architecture laid down by the NeXT team back in the day. NeXTstep now has the dream that MS tried and failed to do back in the day: an OS that runs on every kind of device they make. Funny how that is.
  - joshstrange 4y ago
    I mean they are one of very few companies who have done arch changes like this and they had already done it twice before Rosetta 2. The same engineers might not have been used for all 3 but I'm sure there was at least a tiny bit of overlap between 68k->PPC and PPC->Intel (and likewise overlap between PPC->Intel and Intel->ARM) that coupled with passed down knowledge within the company gives them a leg up. They know the pitfalls, they've see issues/advantages of using certain approaches. I think of it in same way that I've migrated from old->new versions of frameworks/languages in the past with breaking changes and each time I've done it I've gotten better at knowing what to expect, what to look for, places where it makes sense to "just get it working" or "upgrade the code to the new paradigm". The first time or two I did it was as a junior working under senior developers so I wasn't as involved but what did trickle down to me and/or my part in the refactor/upgrade taught me things. Later times when I was in charge (or on my own) I was able to draw on those past experiences. Obviously my work is nowhere near as complicated as arch changes but if you squint and turn your head to the side I think you can see the similarities. > Or were they just lucky to have success both times? I think 2 times might be explained with "luck" but being successful 3 times points to a strong trend IMHO, especially since Rosetta 2 seems to have done even better than Rosetta 1 for the last transition.
karmakaze 4y ago
Vertical integration. My understanding was it's because the Apple silicon ARM has special support to make it fast. Apple has had enough experience to know that some hardware support can go a long way to making the binary emulation situation better.
- saagarjha 4y ago
  That’s not correct, the article goes into details why.
  - nwallin 4y ago
    That is correct, the article goes into details why. See the "Apple's Secret Extension" section as well as the "Total Store Ordering" section. The "Apple's Secret Extension" section talks about how the M1 has 4 flag bits and the x86 has 6 flag bits, and how emulating those 2 extra flags would make every add/sub/cmp instruction significantly slower. Apple has an undocumented extension that adds 2 more flag bits to make the M1's flag bits behave the same as x86. The "Total Store Ordering" section talks about how Apple has added a non-standard store ordering to the M1 than makes the M1 order its stores in the same way x86 guarantees instead of the way ARM guarantees. Without this, there's no good way to translate instructions in code in and around an x86 memory fence; if you see a memory fence in x86 code it's safe to assume that it depends on x86 memory store semantics and if you don't have that you'll need to emulate it with many mostly unnecessary memory fences, which will be devastating for performance.
    - saagarjha 4y ago
      I’m aware of both of these extensions; they’re not actually necessary for most applications. Yes, you trade fidelity with performance, but it’s not that big of a deal. The majority of Rosetta’s performance is good software decisions and not hardware.
      - dougall 4y ago
        Yeah, these features exist, and they help, but I don't think they should be given all the credit. Both "Apple's Secret Extension" and "Total Store Ordering" are features that other emulators can choose to disable to get exactly the same performance. "Apple's Secret Extension" isn't even used by Rosetta 2 on Linux (opting for, at least, explicit parity flag calculations rather than reduced fidelity). It's still fast. TSO is only required for accuracy on multithreaded applications, and the PF and AF flags are basically never used, and, if they are, will usually be used immediately after being set, allowing emulators to achieve reasonable fidelity by only calculating them when used. There's perhaps a better argument for performance-via-vertical-integration with the flag-manipulation extensions, which I believe Apple created and standardised, but which now anyone can use. But the reason I wrote this post is that I think most of the ideas are transferable and could help other emulators :)
superkuh 4y ago
- bogeholm 4y ago
  Thanks for your thoroughly objective insights. I especially appreciate the concrete examples.
  - howinteresting 4y ago
    Here you go for a concrete example: https://news.ycombinator.com/item?id=33493276 https://news.ycombinator.com/item?id=33493276
    - saagarjha 4y ago
      This has nothing to do with Rosetta being incomplete (it has pretty good fidelity).
      - howinteresting 4y ago
        It was direct corroboration of: > Apple users not being able to use the same hardware peripherals or same software as other people is not a problem, it's a feature. There's no doubt the M1/M2 chips are fast. It's just a problem that they're only available in crappy computers that can't run a large amount of software or hardware.
rgiacobazzi 4y ago
Great article!
darzu 4y ago
Does anyone know the names of the key people behind Rosetta 2? In my experience, exceptionally well executed tech like this tends to have 1-2 very talented people leading. I'd like to follow their blog or Twitter.
- trollied 4y ago
  The original Rosetta was written by Transitive, which was formed by spinning a Manchester University research group out. See https://www.software.ac.uk/blog/2016-09-30-heroes-software-engineering-men-and-women-transitive https://www.software.ac.uk/blog/2016-09-30-heroes-software-e... I know a few of their devs went to ARM, some to Apple & a few to IBM (who bought Transitive). I do know a few of their ex staff (and their twitter handles), but I don’t feel comfortable linking them here.
  - scrlk 4y ago
    IIRC the current VP of Core OS at Apple is ex-Manchester/Transitive.
- cwzwarich 4y ago
  I am the creator / main author of Rosetta 2. I don't have a blog or a Twitter (beyond lurking).
  - dougall 4y ago
    Amazing work! It's nice to put a name to it :)
  - darzu 4y ago
    Should you feel inspired to share your learnings, insights, or future ideas about the computing spaces you know, me and I'm sure many other people would be interested to listen! My preferred way to learn about a new (to me) area of tech is to hear the insights of the people who have provably advanced that field. There's a lot of noise to signal in tech blogs.
  - darzu 4y ago
ericbarrett 4y ago
Anybody know if Docker has plans to move from qemu to Rosetta on M1/2 Macs? I've found qemu to be at least 100x slower than the native arch.
titzer 4y ago
Rosetta 2 is great, except it apparently can't run statically-linked (non-PIC) binaries. I am unsure why this limitation exists, but it's pretty annoying because Virgil x86-64-binaries cannot run under Rosetta 2, which means I resort to running on the JVM on my M1...
- saagarjha 4y ago
  Rosetta can run statically linked binaries, but I don’t think anything supports binaries that aren’t relocatable. $ file a.out a.out: Mach-O 64-but executable x86_64 $ tool -L a.out a.out: $ ./a.out Hello, world!
- mirashii 4y ago
  Statically linked binaries are officially unsupported on MacOS in general, so there's no reason to support it on Rosetta either. It's unsupported in MacOS because it assumes binary compatibility on the kernel system call interface, which is not guaranteed.
  - saagarjha 4y ago
    Rosetta was introduced with the promise that it supports binaries that make raw system calls. (And it does indeed support these by hooking the syscall instruction.)
- CharlesW 4y ago
  > Rosetta 2 is great, except it apparently can't run statically-linked (non-PIC) binaries. Interestingly, it supports statically-linked x86 binaries when used with Linux. "Rosetta can run statically linked x86_64 binaries without additional configuration. Binaries that are dynamically linked and that depend on shared libraries require the installation of the shared libraries, or library hierarchies, in the Linux guest in paths that are accessible to both the user and to Rosetta." https://developer.apple.com/documentation/virtualization/running_intel_binaries_in_linux_vms_with_rosetta https://developer.apple.com/documentation/virtualization/run...
- randyrand 4y ago
sedatk 4y ago
TL;DR: One-to-one instruction translation ahead of time instead of complex JIT translations to bet on M1's performance and instruction cache handling.
hot_gril 4y ago
Rosetta 2 has become the poster child for "innovation without deprecation" where I work (not Apple).
- Tijdreiziger 4y ago
  Apple is the king of deprecation, just look at what happened to Rosetta 1 and 32-bit iOS apps.
  - hot_gril 4y ago
    Yes they are, and that makes Rosetta 2 even more special. Though Rosetta 1 got support for 5 years, which is pretty good.
lunixbochs 4y ago
> To see ahead-of-time translated Rosetta code, I believe I had to disable SIP, compile a new x86 binary, give it a unique name, run it, and then run otool -tv /var/db/oah///unique-name.aot (or use your tool of choice – it’s just a Mach-O binary). This was done on old version of macOS, so things may have changed and improved since then. My aotool project uses a trick to extract the AOT binary without root or disabling SIP: https://github.com/lunixbochs/meta/tree/master/utils/aotool https://github.com/lunixbochs/meta/tree/master/utils/aotool
saagarjha 4y ago
One thing that’s interesting to note is that the amount of effort expended here is not actually all that large. Yes, there are smart people working on this, but the performance of Rosetta 2 for the most part is probably the work of a handful of clever people. I wouldn’t be surprised if some of them have an interest in compilers but the actual implementation is fairly straightforward and there isn’t much of the stuff you’d typically see in an optimizing JIT: no complicated type theory or analysis passes. Aside from a handful of hardware bits and some convenient (perhaps intentionally selected) choices in where to make tradeoffs there’s nothing really specifically amazing here. What really makes it special is that anyone (well, any company with a bit of resources) could’ve done it but nobody really did. (But, again, Apple owning the stack and having past experience probably did help them get over the hurdle of actually putting effort into this.)
- dougall 4y ago
  Yeah, agreed. I get the impression it's a small team. But there is a long-tail of weird x86 features that are implemented, that give them amazing compatibility, that I regret not mentioning: * 32-bit support for Wine * full x87 emulation * full SSE2 support (generally converting to efficient NEON equivalents) for performance on SIMD code I consider all of these "compatibility", but that last one in particular should have been in the post, since that's very important to the performance of optimised SIMD routines (plenty of emulators also do SIMD->SIMD, but others just translate SIMD->scalar or SIMD->helper-runtime-call).
  - saagarjha 4y ago
    Guilty!
- menaerus 4y ago
  I think it's about the incentive and not about other companies not doing it. Apple decided to move to ARM and the reason is probably in their strong connection to the ARM ecosystem which basically means that they have an edge with their vertical-integration approach when compared to the other competitors. Apple is one of the three _founding_ companies of ARM. Other two were VLSI Technology and Acorn.
tomcam 4y ago
> Every one-byte x86 push becomes a four byte ARM instruction Can someone explain this to me? I don’t know ARM but it just seems to me a push should not be that expensive.
- dougall 4y ago
  Bit late, and the other comments are right, but it's worth noting that pushes typically aren't that expensive. ARM has a 4-byte STP (store paired) instruction that pushes two values at a time. So usually a push only costs two bytes on ARM, but if you're translating instruction-for-instruction it's four bytes. (I also forgot while writing the post that 2-byte push instructions are common in 64-bit x86 as well. Half the registers can be pushed with a single byte, and the other half require a REX prefix, giving a two-byte push instruction. So even though the quoted statement is true, the difference isn't that bad in general.)
- TazeTSchnitzel 4y ago
  x86 has variable-length instructions, so they can be anything from 1 to 15 bytes long. AArch64 instructions are always 4 bytes long.
- jasonwatkinspdx 4y ago
  The general principle is that RISC style instruction sets are typically fixed length and with only a couple different subformats. Like the prototypical RISC design has one format with an opcode and 3 register fields, and then a second with an opcode and an immediate field. This simplicity and regularity makes the fastest possible decoding hardware much more simple and efficient compared to something like x86 that has a simply dumbfounding number of possible variable length formats. The basic bet of RISC was that larger instruction encodings would be worth it due to the micro architectural advantages they enabled. This more or less was proven out, though the distinction is less distinct today with x86 decoding into uOps and recent ARM standards being quite complex beasts.
kccqzy 4y ago
> The instructions from FEAT_FlagM2 are AXFLAG and XAFLAG, which convert floating-point condition flags to/from a mysterious “external format”. By some strange coincidence, this format is x86, so these instruction are used when dealing with floating point flags. This really made me chuckle. They probably don't want to mention Intel by name, but this just sounds funny. https://developer.arm.com/documentation/100076/0100/A64-Instruction-Set-Reference/A64-General-Instructions/XAFlag https://developer.arm.com/documentation/100076/0100/A64-Inst...
lzooz 4y ago
>The Apple M1 has an undocumented extension that, when enabled, ensures instructions like ADDS, SUBS and CMP compute PF and AF and store them as bits 26 and 27 of NZCV respectively, providing accurate emulation with no performance penalty. If there is no performance penalty why is it implemented as an optional extension?

650 points by pantalaimon 4y ago

The first time I ran into this technology was in the early 90s on the DEC Alpha. They had a tool called "MX" that would translate MIPS Ultrix binaries to Alpha on DEC Unix: https://www.linuxjournal.com/article/1044 https://www.linuxjournal.com/article/1044 Crazy stuff. Rosetta 2 is insanely good. Runs FPS video games even.

deleted 4y ago

[deleted]

0x000xca0xfe 4y ago

I wonder if such a direct translation from ARM to another architecture would even be possible given that the instruction set can be changed at runtime (thumb mode). Does anybody know how often typical ARM32 programs execute this mode switching or if such sections can be recognized statically?

anyfoo 4y ago
If that's really an issue (most code probably sticks to one mode, but I have no data), just translate twice, once in either mode. ¯\_(ツ)_/¯
saagarjha 4y ago
Shouldn’t be hard under this scheme, it tracks indirect branches anyways. Just swap which table you do lookups based on whether you’re in thumb mode or not.
astrange 4y ago
x86 is worse because you can jump into the middle of instructions. In that case you just fall back to JIT though. And luckily AArch64 doesn't have Thumb.

musicale 4y ago

Rosetta 2 is beautiful - I would love it if they kept it as a feature for the long term rather than deprecating it and removing it in the next release of macOS (basically what they did during previous architectural transitions.) If Apple does drop it, maybe they could open source it so it could live on in Linux and BSD at least. ;-) Adding a couple of features to ARM to drastically improve translated x86 code execution sounds like a decent idea - and one that could potentially enable better x86 app performance on ARM Windows as well. I don't know the silicon cost but I'd hope it wasn't dropped in the future.' Thinking a bit larger, I'd also like to see Apple add something like CHERI support to Apple Silicon and macOS to enable efficient memory error checking in hardware. I'd be surprised if they weren't working on something like this already.

webwielder2 4y ago

(Apologies for the flame war quality to this comment, I’m genuinely just expressing an observation) It’s ironic that Apple is often backhandedly complimented by hackers as having “good hardware” when their list of software accomplishments is amongst the most impressive in the industry and contrasts sharply with the best efforts of, say, Microsoft, purportedly a “software company.”

majou 4y ago
The transparent APFS migration was one heck of a feat.
solarkraft 4y ago
Both have pretty impressive engineering. The focus just appears to be different. Apple really cares about user experience (or, maybe, used to), while it seems like Microsoft is trying to (and according to what I hear mostly succeeding) nail the administration experience instead.

fooblaster 4y ago

A lot of the simplicity of this approach relies on the x86 registers being able to be directly mapped to arm registers. This seems to be possible for most x86 registers, even simd registers. Although, I think this falls over for avx512, which is supported on the Mac pro. Arm neon has 32 128 bit registers - avx512 supports 32 512 bit registers and dedicated predicate registers. What do they do? Back to jit mode?

fooblaster 4y ago
oh, they don't support avx: https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment https://developer.apple.com/documentation/apple-silicon/abou...

fulafel 4y ago

For history this was a major milestone in x86 binary translation, Digital FX!32: https://www.usenix.org/legacy/publications/library/proceedings/usenix-nt97/full_papers/chernoff/chernoff.pdf https://www.usenix.org/legacy/publications/library/proceedin... Some apps run faster than the fastest available x86 at the time, and sometimes significantly faster like the Byte benchmark cited above. Of course it helped that the chip was sugnificantly faster than leading x86 chips in the first place.

johnthuss 4y ago

"I believe there’s significant room for performance improvement in Rosetta 2... However, this would come at the cost of significantly increased complexity... Engineering is about making the right tradeoffs, and I’d say Rosetta 2 has done exactly that."

Gigachad 4y ago
Would be a waste of effort when the tool is designed to be obsolete in a few years as everything gets natively compiled.
- solarkraft 4y ago
  They could've amazed a few people a bit more by emulating x86 apps even faster (but M1+Rosetta can already run some stuff faster than an Intel Mac), but then the benefit of releasing native apps would be much decreased ("why bother, it's good enough ..."). It's a delicate political game that they, yet again, seem to be playing pretty well.

MikusR 4y ago

The main reason, M1/2 being incredibly fast. Is listed last.

dagmx 4y ago
Perhaps if you’re comparing against Intel processors, but even on an Apple Silicon Mac, Rosetta 2 vs native versions of apps are no slouch. 20% overhead for a non-native executable is very commendable.
postalrat 4y ago
That isn't the main reason. If Rosetta ran x86 code at 10% the speed of native nobody would be calling it fast.
Someone 4y ago
I don’t think that’s the main reason. The article lists a few things that, I think the main reason is that they made several parts of the CPU behave identical to x86. The M1 and M2 chips: - can be told to do total store ordering, just as x86 does - have of a few status flags that x86 has, but regular arm doesn’t - can be told to make the FPU behave exactly as the x86 FPU It also helps that ARM has many more registers than x86. Because of that the emulator can map the x86 registers to ARM registers, and have registers to spare for use by the emulator.
- astrange 4y ago
  None of those custom features are used in Linux/in VMs, and it's still fairly fast.

Vt71fcAqt7 4y ago

I hope Rosetta is here to stay and continues developement. And I hope what is learned from it can be used to make a RISC-V version of it. translating native ARM to RISC-V should be much easier than x86 to ARM as I understand it, so one could conceivably do x86 -> ARM -> RISC-V.

masklinn 4y ago
> I hope Rosetta is here to stay and continues developement. It almost certainly is not. Odds are Apple will eventually remove Rosetta II, as they did Rosetta back in the days, once they consider the need for that bridge to be over (Rosetta was added in 2006 in 10.4, and removed in 2011 from 10.7). > And I hope what is learned from it can be used to make a RISC-V version of it. translating native ARM to RISC-V should be much easier than x86 to ARM as I understand it, so one could conceivably do x86 -> ARM -> RISC-V. That's not going to happen unless Apple decides to switch from ARM to RISC-V, and... why would they? They've got 15 years experience and essentially full control on ARM.
- quux 4y ago
  It would be funny/not funny if in a few years Apple removes Rosetta 2 for Mac apps but keeps the Linux version forever so docker can run at reasonable speeds.
- stu2b50 4y ago
  Rosetta 1 had a ticking time bomb. Apple was licensing it from a 3rd party. Rosetta 2 is all in house as far as we know. Different CEO as well. Jobs was more opinionated on “principles” - Cook is more than happy to sell what people will buy. I think Rosetta 2 will last.
  - deleted 4y ago
    [deleted]
  - TillE 4y ago
    What important Intel-only macOS software is going to exist in five years? It's basically only games and weird tiny niches, and Apple is pretty happy to abandon both those categories. The saving grace is that there's very few interesting Mac-exclusive games in the Intel era.

hinkley 4y ago

Apple is doing some really interesting but really quiet work in the area of VMs. I feel like we don’t give them enough credit but maybe they’ve put themselves in that position by not bragging enough about what they do. As a somewhat related aside, I have been watching Bun (low startup time Node-like on top of Safari’s JavaScript engine) with enough interest that I started trying to fix a bug, which is somewhat unusual for me. I mostly contribute small fixes to tools I use at work. I can’t quite grok Zig code yet so I got stuck fairly quickly. The “bug” turned out to be default behavior in a Zig stdlib, rather than in JavaScript code. The rest is fairly tangential but suffice it to say I prefer self hosted languages but this probably falls into the startup speed compromise. Being low startup overhead makes their VM interesting, but the fact that it benchmarks better than Firefox a lot of the time and occasionally faster than v8 is quite a bit of quiet competence.

jraph 4y ago
> feel like we don’t give them enough credit but maybe they’ve put themselves in that position by not bragging enough about what they do. And maybe also by keeping the technology closed and Apple-specific. Many people who could be interested in using it don't have access to it.
- jolux 4y ago
  WebKit B3 is open source: https://webkit.org/docs/b3/ https://webkit.org/docs/b3/
  - deleted 4y ago
    [deleted]
- freedomben 4y ago
  Exactly. As someone who would be very interested in this, but don't use Apple products, it's just not exciting because it's not accessible to me (I can't even test it as a user). If they wanted to write a whitepaper about it to share knowledge, that might be interesting, but given that it's Apple I'm not gonna hold my breath.

peatmoss 4y ago

Not having any particular domain experience here, I've idly wondered whether or not there's any role for neural net models in translating code for other architectures. We have giant corpuses of source code, compiled x86_64 binaries, and compiled arm64 binaries. I assume the compiled binaries represent approximately our best compiler technology. It seems predicting an arm binary from an x86_64 binary would not be insane? If someone who actually knows anything here wants to disabuse me of my showerthoughts, I'd appreciate being able to put the idea out of my head :-)

qsort 4y ago
You would need a hybrid architecture with a NN generating guesses and a "watchdog" shutting down errors. Neural models are basically universal approximators. Machine code needs to be obscenely precise to work. Unless you're doing something else in the backend, it's just a turbo SIGILL generator.
- throw10920 4y ago
  This is all true - machine code needs to be "basically perfect" to work. However, there are lots of problems in CS that are easier to check the answer to a solution than to solve in the first place. It may turn out to be the case that a well-tuned model can quickly produce solutions to some code-generation problems, that those solutions have a high enough likelihood of being correct, that it's fast enough to check (and maybe try again), and that this entire process is faster than state-of-the-art classical algorithms. However, if that were the case, I might also expect us to be able to extract better algorithms from the model - intuitively, machine code generation "feels" like something that's just better implemented through classical algorithms. Have you met a human that can do register allocation faster than LLVM?
- classichasclass 4y ago
  > turbo SIGILL generator This gave me the delightful mental image of a CPU smashing headlong into a brick wall, reversing itself, and doing it again. Which is pretty much what this would do.
4y ago

menaerus 4y ago

> Rosetta 2 translates the entire text segment of the binary from x86 to ARM up-front. Do I understand correctly that the Rosetta is basically a transpiler from x86-64 machine code to ARM machine code which is run prior to the binary execution? If so, does it affect the application startup times?

nilsb 4y ago
Yes, it does. The delay of the first start of an app is quite noticeable. But the transpiled binary is apparently cached somewhere.
- saagarjha 4y ago
  /var/db/oah.
nicoburns 4y ago
> If so, does it affect the application startup times? It does, but only the very first time you run the application. The result of the transpilation is cached so it doesn't have to be computed again until the app is updated.
- dylan604 4y ago
  Does that essentially mean each non-native app is doubled in disk use? Maybe not doubled but requires more space to be sure.
  - saagarjha 4y ago
    Yes.
  - varenc 4y ago
    Yes... you can see the cache in /var/db/oah/ Though only the actual binary size that gets doubled. For large apps it’s usually not the binary that’s taking up most of the space.
- 4y ago

hinkley 4y ago

I remember years ago when Java adjacent research was all the rage, HP had a problem that was “Rosetta lite” if you will. They had a need to run old binaries on new hardware that wasn’t exactly backward compatible. They made a transpiler that worked on binaries. It might have even been a JIT but that part of the memory is fuzzy. What made it interesting here was that as a sanity check they made an A->A mode where they took in one architecture and spit out machine code for the same architecture. The output was faster than the input. Meaning that even native code has some room for improvement with JIT technology. I have been wishing for years that we were in a better place with regard to compilers and NP complete problems where the compilers had a fast mode for code-build-test cycles and a very slow incremental mode for official builds. I recall someone telling me the only thing they liked about the Rational IDE (C and C++?) was that it cached precompiled headers, one of the Amdahl’s Law areas for compilers. If you changed a header, you paid the recompilation cost and everyone else got a copy. I love whenever the person that cares about something gets to pay the consequence instead of externalizing it on others. And having some CI machines or CPUs that just sit around chewing on Hard Problems all day for that last 10% seems to be to be a really good use case in a world that’s seeing 16 core consumer hardware. Also caching hints from previous runs is a good thing.

fuckstick 4y ago
> The output was faster than the input. So if you ran the input back through the output multiple times then that means you could eventually get the runtime down to 0.
- deleted 4y ago
  [deleted]
- avidiax 4y ago
  Probably the output of the decade-old compiler that produced the original binary had no optimizations.
  - hinkley 4y ago
    That too but the eternal riddle of optimizer passes is which ones reveal structure and which obscure it. Do I loop unroll or strength reduce first? If there are heuristics about max complexity for unrolling or inlining then it might be “both”. And then there’s processor family versus this exact model.

dynjo 4y ago

It is quite astonishing how seamless Apple has managed to make the Intel to ARM transition, there are some seriously smart minds behind Rosetta. I honestly don't think I had a single software issue during the transition!

perardi 4y ago
I think the end of support for 32-bit applications in 2019 helped, slightly, with the run-up. Assuming you weren’t already shipping 64-bit applications…which would be weird…updating the application probably required getting everything into a contemporary version of Xcode, cleaning out the cruft, and getting it compiling nice and cleanly. After that, the ARM transition was kind of a “it just works” scenario. Now, I’m sure Adobe and other high-performance application developers had to do some architecture-specific tweaks, but, gotta think Apple clued them in ahead of time as to what was coming.
wombat-man 4y ago
There's an annoying dwarf fortress bug but other than that, same
ChuckNorris89 4y ago
If that blows your mind, you should see how Microsoft did the emulation of the PowerPC based Xeon chip to X86 so you can play Xbox 360 games on Xbox One. There's an old pdf from Microsoft researchers with the details but I can't seem to find it right now.
- dev_tty01 4y ago
  They also bought a boat load of PPC Macs for much of the early Xbox dev work.
- RedShift1 4y ago
  Any good videos on that?
poulpy123 4y ago
having total control on the hardware and the software didn't hurt for sure

iainmerrick 4y ago

This is a great writeup. What a clever design! I remember Apple had a totally different but equally clever solution back in the days of the 68K-to-PowerPC migration. The 68K had 16-bit instruction words, usually with some 16-bit arguments. The emulator’s core loop would read the next instruction and branch directly into a big block of 64K x 8 bytes of PPC code. So each 68K instruction got 2 dedicated PPC instructions, typically one to set up a register and one to branch to common code. What that solution and Rosetta 2 have in common is that they’re super pragmatic - fast to start up, with fairly regular and predictable performance across most workloads, even if the theoretical peak speed is much lower than a cutting-edge JIT. Anyone know how they implemented PPC-to-x86 translation?

klelatti 4y ago
That’s really interesting. You might enjoy reading about the VM embedded into the Busicom calculator that used the Intel 4004 [1] They squeezed a virtual machine with 88 instructions into less than 1k of memory! [1] https://thechipletter.substack.com/p/bytecode-and-the-busicom https://thechipletter.substack.com/p/bytecode-and-the-busico...
- hoosieree 4y ago
  And here I was feeling impressed with myself for implementing the Nand2Tetris VM translator in ~2k of python... wow. Respect for the elders!
- iainmerrick 4y ago
  That is nifty! Sounds very similar to a Forth interpreter.
  - deleted 4y ago
    [deleted]
  - DonHopkins 4y ago
    There's also OpenFirmware's platform independent Forth bytecode "FCode": https://en.wikipedia.org/wiki/Open_Firmware https://en.wikipedia.org/wiki/Open_Firmware >Open Firmware Forth Code may be compiled into FCode, a bytecode which is independent of instruction set architecture. A PCI card may include a program, compiled to FCode, which runs on any Open Firmware system. In this way, it can provide boot-time diagnostics, configuration code, and device drivers. FCode is also very compact, so that a disk driver may require only one or two kilobytes. Therefore, many of the same I/O cards can be used on Sun systems and Macintoshes that used Open Firmware. FCode implements ANS Forth and a subset of the Open Firmware library.

retskrad 4y ago

Apple Silicon will be Tim Cook's legacy.

jeffbee 4y ago

I wonder how much hand-tuning there is in Rosetta 2 for known, critical routines. One of the tricks Transmeta used to get reasonable performance on their very slow Crusoe CPU was to recognize critical Windows functions and replace them with a library of hand-optimized native routines. Of course that's a little different because Rosetta 2 is targeting an architecture that is generally speaking at least as fast as the x86 architecture it is trying to emulate, and that's been true for most cross-architecture translators historically like DEC's VEST that ran VAX code on Alpha, but Transmeta CMS was trying to target a CPU that was slower.

saagarjha 4y ago
Haven’t spotted any in particular.
- dougall 4y ago
  Yeah, I haven't either, but I haven't looked. So, I'm not sure, but I wouldn't expect any "function recognition" tricks, since there isn't really static linking, but I would expect the e.g. memcpy and strcpy implementations in the ahead-of-time translated shared cache to be written in arm64 assembly rather than translated.
  - saagarjha 4y ago
    Indeed, see the _platform_*$VARIANT$Rosetta implementations in libsystem_platform.dylib.

pjmlp 4y ago

Back in the early days of Windows NT everywhere, the Alpha version had a similar JIT emulation.

agentcooper 4y ago

I am interested in this domain, but lacking knowledge to fully understand the post. Any recommendations on good books/courses/tutorials related to low level programming?

saagarjha 4y ago
I’d recommend going through a compilers curriculum, then reading up on past binary translation efforts.

pjmlp 4y ago

Back in the early days of Windows NT everywhere, the Alpha version had a similar JIT emulation. https://en.m.wikipedia.org/wiki/FX!32 https://en.m.wikipedia.org/wiki/FX!32 Or for a more technical deep dive, https://www.usenix.org/publications/library/proceedings/usenix-nt97/full_papers/chernoff/chernoff.pdf https://www.usenix.org/publications/library/proceedings/usen...

mosburger 4y ago
OMG I forgot about FX!32. My first co-op was as a QA tester for the DEC Multia, which they moved from the Alpha processor to Intel midway through. I did a skunkworks project for the dev team attempting to run the newer versions of Multia's software (then Intel-based) on older Alpha Multias using FX!32. IIRC it was still internal use only/beta, but it worked quite well!

manv1 4y ago

Apple's historically been pretty good at making this stuff. Their first 68k -> PPC emulator (Davidian's) was so good that for some things the PPC Mac was the fastest 68k mac you could buy. The next-gen DR emulator (and SpeedDoubler etc) made things even faster. I suspect the ppc->x86 stuff was slower because x86 just doesn't have the registers. There's only so much you can do.

Wowfunhappy 4y ago
> Apple's historically been pretty good at making this stuff. Their first 68k -> PPC emulator (Davidian's) was so good that for some things the PPC Mac was the fastest 68k mac you could buy. Not arguing the facts here, but I'm curious—are these successes related? And if so, how has Apple done that? I would imagine that very few of the engineers who programmed Apple's 68k emulator are still working at Apple today. So, why is Apple still so good at this? Strong internal documentation? Conducive management practices? Or were they just lucky both times?
- manv1 4y ago
  I suspect that there's a design document somewhere that has secrets and tips. Plus MacOS and NeXTstep have been ported so many times that by now any arch-level things are well-isolated. A lot of the pain of MacOS on x86 were related to kernel extensions and driver API changes. And a lot of stuff fell out during the 64-bit transition. In general, MacOS/iOS/iPadOS owe everything to the excellent architecture laid down by the NeXT team back in the day. NeXTstep now has the dream that MS tried and failed to do back in the day: an OS that runs on every kind of device they make. Funny how that is.
- joshstrange 4y ago
  I mean they are one of very few companies who have done arch changes like this and they had already done it twice before Rosetta 2. The same engineers might not have been used for all 3 but I'm sure there was at least a tiny bit of overlap between 68k->PPC and PPC->Intel (and likewise overlap between PPC->Intel and Intel->ARM) that coupled with passed down knowledge within the company gives them a leg up. They know the pitfalls, they've see issues/advantages of using certain approaches. I think of it in same way that I've migrated from old->new versions of frameworks/languages in the past with breaking changes and each time I've done it I've gotten better at knowing what to expect, what to look for, places where it makes sense to "just get it working" or "upgrade the code to the new paradigm". The first time or two I did it was as a junior working under senior developers so I wasn't as involved but what did trickle down to me and/or my part in the refactor/upgrade taught me things. Later times when I was in charge (or on my own) I was able to draw on those past experiences. Obviously my work is nowhere near as complicated as arch changes but if you squint and turn your head to the side I think you can see the similarities. > Or were they just lucky to have success both times? I think 2 times might be explained with "luck" but being successful 3 times points to a strong trend IMHO, especially since Rosetta 2 seems to have done even better than Rosetta 1 for the last transition.

karmakaze 4y ago

Vertical integration. My understanding was it's because the Apple silicon ARM has special support to make it fast. Apple has had enough experience to know that some hardware support can go a long way to making the binary emulation situation better.

saagarjha 4y ago
That’s not correct, the article goes into details why.
- nwallin 4y ago
  That is correct, the article goes into details why. See the "Apple's Secret Extension" section as well as the "Total Store Ordering" section. The "Apple's Secret Extension" section talks about how the M1 has 4 flag bits and the x86 has 6 flag bits, and how emulating those 2 extra flags would make every add/sub/cmp instruction significantly slower. Apple has an undocumented extension that adds 2 more flag bits to make the M1's flag bits behave the same as x86. The "Total Store Ordering" section talks about how Apple has added a non-standard store ordering to the M1 than makes the M1 order its stores in the same way x86 guarantees instead of the way ARM guarantees. Without this, there's no good way to translate instructions in code in and around an x86 memory fence; if you see a memory fence in x86 code it's safe to assume that it depends on x86 memory store semantics and if you don't have that you'll need to emulate it with many mostly unnecessary memory fences, which will be devastating for performance.
  - saagarjha 4y ago
    I’m aware of both of these extensions; they’re not actually necessary for most applications. Yes, you trade fidelity with performance, but it’s not that big of a deal. The majority of Rosetta’s performance is good software decisions and not hardware.
    - dougall 4y ago
      Yeah, these features exist, and they help, but I don't think they should be given all the credit. Both "Apple's Secret Extension" and "Total Store Ordering" are features that other emulators can choose to disable to get exactly the same performance. "Apple's Secret Extension" isn't even used by Rosetta 2 on Linux (opting for, at least, explicit parity flag calculations rather than reduced fidelity). It's still fast. TSO is only required for accuracy on multithreaded applications, and the PF and AF flags are basically never used, and, if they are, will usually be used immediately after being set, allowing emulators to achieve reasonable fidelity by only calculating them when used. There's perhaps a better argument for performance-via-vertical-integration with the flag-manipulation extensions, which I believe Apple created and standardised, but which now anyone can use. But the reason I wrote this post is that I think most of the ideas are transferable and could help other emulators :)

superkuh 4y ago

bogeholm 4y ago
Thanks for your thoroughly objective insights. I especially appreciate the concrete examples.
- howinteresting 4y ago
  Here you go for a concrete example: https://news.ycombinator.com/item?id=33493276 https://news.ycombinator.com/item?id=33493276
  - saagarjha 4y ago
    This has nothing to do with Rosetta being incomplete (it has pretty good fidelity).
    - howinteresting 4y ago
      It was direct corroboration of: > Apple users not being able to use the same hardware peripherals or same software as other people is not a problem, it's a feature. There's no doubt the M1/M2 chips are fast. It's just a problem that they're only available in crappy computers that can't run a large amount of software or hardware.

rgiacobazzi 4y ago

Great article!

darzu 4y ago

Does anyone know the names of the key people behind Rosetta 2? In my experience, exceptionally well executed tech like this tends to have 1-2 very talented people leading. I'd like to follow their blog or Twitter.

trollied 4y ago
The original Rosetta was written by Transitive, which was formed by spinning a Manchester University research group out. See https://www.software.ac.uk/blog/2016-09-30-heroes-software-engineering-men-and-women-transitive https://www.software.ac.uk/blog/2016-09-30-heroes-software-e... I know a few of their devs went to ARM, some to Apple & a few to IBM (who bought Transitive). I do know a few of their ex staff (and their twitter handles), but I don’t feel comfortable linking them here.
- scrlk 4y ago
  IIRC the current VP of Core OS at Apple is ex-Manchester/Transitive.
cwzwarich 4y ago
I am the creator / main author of Rosetta 2. I don't have a blog or a Twitter (beyond lurking).
- dougall 4y ago
  Amazing work! It's nice to put a name to it :)
- darzu 4y ago
  Should you feel inspired to share your learnings, insights, or future ideas about the computing spaces you know, me and I'm sure many other people would be interested to listen! My preferred way to learn about a new (to me) area of tech is to hear the insights of the people who have provably advanced that field. There's a lot of noise to signal in tech blogs.
- darzu 4y ago

ericbarrett 4y ago

Anybody know if Docker has plans to move from qemu to Rosetta on M1/2 Macs? I've found qemu to be at least 100x slower than the native arch.

titzer 4y ago

Rosetta 2 is great, except it apparently can't run statically-linked (non-PIC) binaries. I am unsure why this limitation exists, but it's pretty annoying because Virgil x86-64-binaries cannot run under Rosetta 2, which means I resort to running on the JVM on my M1...

saagarjha 4y ago
Rosetta can run statically linked binaries, but I don’t think anything supports binaries that aren’t relocatable. $ file a.out a.out: Mach-O 64-but executable x86_64 $ tool -L a.out a.out: $ ./a.out Hello, world!
mirashii 4y ago
Statically linked binaries are officially unsupported on MacOS in general, so there's no reason to support it on Rosetta either. It's unsupported in MacOS because it assumes binary compatibility on the kernel system call interface, which is not guaranteed.
- saagarjha 4y ago
  Rosetta was introduced with the promise that it supports binaries that make raw system calls. (And it does indeed support these by hooking the syscall instruction.)
CharlesW 4y ago
> Rosetta 2 is great, except it apparently can't run statically-linked (non-PIC) binaries. Interestingly, it supports statically-linked x86 binaries when used with Linux. "Rosetta can run statically linked x86_64 binaries without additional configuration. Binaries that are dynamically linked and that depend on shared libraries require the installation of the shared libraries, or library hierarchies, in the Linux guest in paths that are accessible to both the user and to Rosetta." https://developer.apple.com/documentation/virtualization/running_intel_binaries_in_linux_vms_with_rosetta https://developer.apple.com/documentation/virtualization/run...
randyrand 4y ago

sedatk 4y ago

TL;DR: One-to-one instruction translation ahead of time instead of complex JIT translations to bet on M1's performance and instruction cache handling.

hot_gril 4y ago

Rosetta 2 has become the poster child for "innovation without deprecation" where I work (not Apple).

Tijdreiziger 4y ago
Apple is the king of deprecation, just look at what happened to Rosetta 1 and 32-bit iOS apps.
- hot_gril 4y ago
  Yes they are, and that makes Rosetta 2 even more special. Though Rosetta 1 got support for 5 years, which is pretty good.

lunixbochs 4y ago

> To see ahead-of-time translated Rosetta code, I believe I had to disable SIP, compile a new x86 binary, give it a unique name, run it, and then run otool -tv /var/db/oah///unique-name.aot (or use your tool of choice – it’s just a Mach-O binary). This was done on old version of macOS, so things may have changed and improved since then. My aotool project uses a trick to extract the AOT binary without root or disabling SIP: https://github.com/lunixbochs/meta/tree/master/utils/aotool https://github.com/lunixbochs/meta/tree/master/utils/aotool

saagarjha 4y ago

One thing that’s interesting to note is that the amount of effort expended here is not actually all that large. Yes, there are smart people working on this, but the performance of Rosetta 2 for the most part is probably the work of a handful of clever people. I wouldn’t be surprised if some of them have an interest in compilers but the actual implementation is fairly straightforward and there isn’t much of the stuff you’d typically see in an optimizing JIT: no complicated type theory or analysis passes. Aside from a handful of hardware bits and some convenient (perhaps intentionally selected) choices in where to make tradeoffs there’s nothing really specifically amazing here. What really makes it special is that anyone (well, any company with a bit of resources) could’ve done it but nobody really did. (But, again, Apple owning the stack and having past experience probably did help them get over the hurdle of actually putting effort into this.)

dougall 4y ago
Yeah, agreed. I get the impression it's a small team. But there is a long-tail of weird x86 features that are implemented, that give them amazing compatibility, that I regret not mentioning: * 32-bit support for Wine * full x87 emulation * full SSE2 support (generally converting to efficient NEON equivalents) for performance on SIMD code I consider all of these "compatibility", but that last one in particular should have been in the post, since that's very important to the performance of optimised SIMD routines (plenty of emulators also do SIMD->SIMD, but others just translate SIMD->scalar or SIMD->helper-runtime-call).
- saagarjha 4y ago
  Guilty!
menaerus 4y ago
I think it's about the incentive and not about other companies not doing it. Apple decided to move to ARM and the reason is probably in their strong connection to the ARM ecosystem which basically means that they have an edge with their vertical-integration approach when compared to the other competitors. Apple is one of the three _founding_ companies of ARM. Other two were VLSI Technology and Acorn.

tomcam 4y ago

> Every one-byte x86 push becomes a four byte ARM instruction Can someone explain this to me? I don’t know ARM but it just seems to me a push should not be that expensive.

dougall 4y ago
Bit late, and the other comments are right, but it's worth noting that pushes typically aren't that expensive. ARM has a 4-byte STP (store paired) instruction that pushes two values at a time. So usually a push only costs two bytes on ARM, but if you're translating instruction-for-instruction it's four bytes. (I also forgot while writing the post that 2-byte push instructions are common in 64-bit x86 as well. Half the registers can be pushed with a single byte, and the other half require a REX prefix, giving a two-byte push instruction. So even though the quoted statement is true, the difference isn't that bad in general.)
TazeTSchnitzel 4y ago
x86 has variable-length instructions, so they can be anything from 1 to 15 bytes long. AArch64 instructions are always 4 bytes long.
jasonwatkinspdx 4y ago
The general principle is that RISC style instruction sets are typically fixed length and with only a couple different subformats. Like the prototypical RISC design has one format with an opcode and 3 register fields, and then a second with an opcode and an immediate field. This simplicity and regularity makes the fastest possible decoding hardware much more simple and efficient compared to something like x86 that has a simply dumbfounding number of possible variable length formats. The basic bet of RISC was that larger instruction encodings would be worth it due to the micro architectural advantages they enabled. This more or less was proven out, though the distinction is less distinct today with x86 decoding into uOps and recent ARM standards being quite complex beasts.

kccqzy 4y ago

> The instructions from FEAT_FlagM2 are AXFLAG and XAFLAG, which convert floating-point condition flags to/from a mysterious “external format”. By some strange coincidence, this format is x86, so these instruction are used when dealing with floating point flags. This really made me chuckle. They probably don't want to mention Intel by name, but this just sounds funny. https://developer.arm.com/documentation/100076/0100/A64-Instruction-Set-Reference/A64-General-Instructions/XAFlag https://developer.arm.com/documentation/100076/0100/A64-Inst...

lzooz 4y ago

>The Apple M1 has an undocumented extension that, when enabled, ensures instructions like ADDS, SUBS and CMP compute PF and AF and store them as bits 26 and 27 of NZCV respectively, providing accurate emulation with no performance penalty. If there is no performance penalty why is it implemented as an optional extension?