7 ms·
Pushing the limits of RISC-V emulation
- dmitrygr 2mo agoAt the end it is not emulation but static recompilation (which is, arguably, cooler)
- throwaway27448 2mo agoHow do you differentiate the two? What is the benefit of such a distinction? Why not use eg threaded emulation vs recompiled emulation?
- dmitrygr 2mo agoEmulation visits instructions as they are executed. Static recompilation will (at translation time) visit instructions that can be discovered, even if they never run. eg: if (rand64() == 0x123456789abcdef0ull) baz = bar; an emulator will likely never visit that assignment. A static recompiler will translate it.
- throwaway27448 2mo agoHm. What is the utility of this distinction? Anyway, qemu certainly seems like it would fall under your definition of "emulator" despite obviously dynamically recompiling.
- dmitrygr 2mo agowhich is why i said "static recompiler" and not just "recompiler" distinction is that an emulator is much simpler, while a static recompiler is a lot more work and thus ~30% more cool
- throwaway27448 2mo agoAh. I admit I've only ever made dynamic recompilers
- monocasa 2mo agoIDK, a lot of the emulators I've seen and a couple I've written will visit that. Not everything is built on simple traces, but a lot of the time will translate more complex graphs at a time.
- drunken_thor 2mo agoIn my limited understanding, static recompiling is like JIT transpilation from one arch to another where emulation runs each instruction calling behaviour depending on the instruction. As for tradeoffs my knowledge is not wide enough to declare anything certain.
- throwaway27448 2mo agoWhat is the distinction in your mind? Can recompiling not call each instruction? Can threaded interpretation not perform optimization?
- shuklaayush 2mo agoYeah, I went back and forth on the title. It's definitely not an interpreter. I still think of emulation as the umbrella term though, running code for one architecture on another, with interpretation and dynamic/static recompilation being different ways to get there
- beholdo 2mo agoThe coolest interpreter technique I saw was one that put instruction bodies in static functions which the "compiler" main loop would memcpy the body of the function out to straight-line code that would be executed from memory - a poor man's jit. All instruction functions had the same args and gcc would emit position independent code with the same predictable register calling convention. Brittle as hell, sure, but great compilation speed with low run-time overhead. It was able to run interpreted code at 1/5th of compiled code speed, compared to 1/10th speed for typical highly optimized computed goto loop interpreters. I can't find a link, but if anyone recalls or wrote such an jit interpreter, please post.
- scheme271 2mo agoSounds like a copy and patch JIT ( https://en.wikipedia.org/wiki/Copy-and-patch https://en.wikipedia.org/wiki/Copy-and-patch ). The python interpreter was experimenting with this and it provides a decent speedup.
- ack_complete 2mo agoOdd that the Wikipedia article gives 2021 as the first description of this technique when it is far, far older than that. I worked with a software rasterizer JIT that used it in ~2003 and I thought similar techniques were used in the classic MacOS m68k emulator on PowerPC.
- scheme271 2mo agoYeah, I think it was in use before. The wiki article is probably incorrect in the 2021 date.
- kijiki 2mo agoqemu used to use that technique, but as you note, it was pretty brittle. They switched to the more traditional TCG backend.
- shuklaayush 2mo agoNeat, I guess this gets you the same output as a per-instruction translator without having to run an assembler and linker over the whole program. I should try this and add it to the post for completeness
- oso2k 2mo agoI was not impressed by author’s surprise that original emulation ran at 1/10th speed. For the past 50 years, that’s usually the target for calling (“unaccelerated”) emulation good enough. Especially for architectures that are current or in development or otherwise require some sort of instruction translation. The last 30 years we’ve seen innovations like dynamic recompilation, fat binaries, JIT VMs, and profile guided optimization. So that has created an expectation of sub-order of magnitude run time performance. It’s a fanciful time we live in.
- shuklaayush 2mo agoHey, author here. That's fair, I didn't have the background. I knew interpreters would be slow, I just didn't expect it to still be ~10x after all the optimizations. After reading about it more though, that does seem to be the norm
- MiroslavPokorny 2mo agoCPU architectures evolve and often get replaces, eg Mac has moved from 68k -> Power -> x86 -> ARM or even Windows PC have also jumped from 32 to 64 bits and the same is also true of SIMD. Why dont O/S support executables with something like LLVM binaries and generate the native code at load time ? This would solve so many problems including the need for emulators, because all binaries would work on all CPUS, and the OS would produce the best code at load time. No more cpu detection, vector stuff always works on the latest & widest instructions that are available. CPUs can also retired old legacy instructions without worry and more.
- postalrat 2mo agoWould that support self modifying code?
- pjmlp 2mo agoSelf modifying code is frowned upon on modern platforms, due to its security implications.
- MiroslavPokorny 2mo agoSelf modifying code is not allowed on some o/s because pages with code are marked as non writable, because bad things can happen when code is writable, eg buffer overflows.
- joha4270 2mo agoThis does exist, we call it Java (or C# or probably a dozen other implementations) The big tradeoff you're making is that you have significantly less time to run your optimizer, since not everybody has a beefy machine or the patience to wait a day for their browser to start first time. You could try doing optimization ahead of time, but I think (I could be wrong here) you would inevitably end up adding in some CPU assumptions if you went much further. This also somewhat conflicts with an advantage of VM based execution, that new optimizations apply to old binaries. I'll also note that hand rolled assembly/SIMD code still beats compilers at the extreme end and you would either have to throw that away, or get all the disadvantages mentioned above without all the advantages
- shuklaayush 2mo agoAuthor here, thanks for all the comments, didn't expect this to get picked up. I've been working on speeding up RISC-V emulation for work and wrote this up as I went. Still learning this space, so I'd be keen to hear from people who've worked on emulators or binary translation, especially where you think this approach falls short