4 ms·

The other day I came across an interesting "alternative" to WASM which gives you OS portability using fully native code, without CPU portability, the latter see

The other day I came across an interesting "alternative" to WASM which gives you OS portability using fully native code, without CPU portability, the latter seeming not that big of a deal these days anyway as cross compilers have got quite good and there are only two CPU archs in wide usage anyway. The idea is to simply run normal Linux binaries on macOS and Windows. How? You create a virtual machine using the Mac/Windows APIs without any OS inside, in fact without even any virtual hardware. It's literally just a new address space and some trivial min-viable VM configuration. Then you map the ELF binary and a ld.so into the VM with a minimal ELF interpreter, kick off execution and anytime there's a syscall you trap it and translate to the host OS syscalls. It can work quite well on macOS because the syscall interface is so similar. Note that this sort of VM is not: • A sandbox • A hardware abstraction Apps run this way hold all their data in the filing system of the host OS, they use the network stack of the host OS, etc. The VM is only being used to allow trapping and emulation of the syscall interface. The app isn't aware that it's being run in a special CPU mode on top of an emulated kernel. Advantages: lightweight, simple, apps can use all CPU features, can run at native speed, the Linux syscall interface is highly stable, based on POSIX specifications and you can easily pick a subset of it to standardize. Disadvantages: requires the emulator, apps exposed to missing features or quirks of the host OS e.g. Windows file system performance is much lower than Linux. WSL1 sort of worked that way, albeit without the VM aspect that lets userspace apps do it. They abandoned it partly for performance reasons and users expected all existing Linux apps to just work. But WASM doesn't target existing apps. It expects developers to bend and do things the WASM way, and accepts that not all apps are compatible with it, so that's not necessarily a problem. An example of how to implement this is NOAH: https://github.com/linux-noah/noah/ https://github.com/linux-noah/noah/

mwcampbell 4y ago
> without CPU portability, the latter seeming not that big of a deal these days anyway as cross compilers have got quite good and there are only two CPU archs in wide usage anyway Do we really want to surrender to the dominance of two proprietary ISAs? To me, the insistence on something like wasm bytecode is a kind of optimism, a hope that the current duopoly won't be permanent.
- mike_hearn 4y ago
  Well, Apple have proven several times now that you don't need a bytecode abstraction to allow CPU architecture transitions. They went from PowerPC to Intel to ARM whilst preserving the benefits of native code each time, using relatively simple stuff that stayed out of developers way and didn't require buyin to entirely new ways of coding or APIs. Rosetta 2 is just an ordinary AOT transpiler with a few magic flags in their ARM fork that lets them optionally match Intel semantics for code that needs it, other people should be able to match that. So bytecode is useful for lots of other reasons when paired with good runtimes, but it doesn't seem to be necessary to let you switch CPU ISAs or vendors. Meanwhile the costs are super high. WASM doesn't and isn't going to expose the real capabilities of the hardware any time soon, so it'll always lag behind in performance. The web has an unfortunate design goal of killing all the incentives OS and hardware makers have to add new features, because browser makers are these days pretty averse to allowing web pages to have special capabilities on specific underlying platforms, so features only get exposed to the web if every (major) OS and CPU combination supports them. But why add new features if apps will only start to use them if all your competitors also ship the same feature - there's no competitive advantage to be had there, which in turn means there's no point in investing in R&D. In turn that means that there's actually no point in designing or deploying a new ISA if everything is WASM. Why bother if it can't add any unique capabilities or nothing would use them? RISC-V says licensing is a good enough reason, but not many competitive design houses seem to care: making a high performance CPU is so expensive that ARM licensing costs just aren't a big deal in the grand scheme of things and in practice silicon OEMs seem fine with outsourcing ISA development to them. Leaning in to native code fixes the incentive problem, and software is constantly being updated these days anyway. If CPU portability was all that mattered it'd be cheaper and quicker to invest in better cross-compilation toolchains, like a standard file format for expressing all the URLs to all the different OS/CPU specific binaries for a program and build servers that could incrementally build and upload binaries for less popular ISAs in the background. I outline a proposal for an alternative web-like thing that works this way here (whilst also supporting wasm and other bytecodes): https://docs.google.com/document/d/1oDBw4fWyRNug3_f5mXWdlgDI4J5AbxVWKEeYr6hscT8/edit?usp=sharing https://docs.google.com/document/d/1oDBw4fWyRNug3_f5mXWdlgDI...
  - 4y ago
evmar 4y ago
This (without a VM) is more or less what Wine does, but implementing the Windows ABI instead of the Linux one.
jakobson14 4y ago
It's astounding how few people know that this already exists, and has for years and years, in the from of user-mode QEMU. https://qemu-project.gitlab.io/qemu/user/index.html https://qemu-project.gitlab.io/qemu/user/index.html This is, by the way, how Hangover is supposed to work.
- mike_hearn 4y ago
  You're right, I didn't know about that. I thought QEMU is always a full OS emulator. Could you give a link for hangover? I'm not sure how to disambiguate in search from actual hangovers.
  - jakobson14 4y ago
    https://github.com/AndreRH/hangover https://github.com/AndreRH/hangover https://www.phoronix.com/news/Wine-Hangover-2023-Open-Source https://www.phoronix.com/news/Wine-Hangover-2023-Open-Source https://www.phoronix.com/news/Hangover-0.8.3 https://www.phoronix.com/news/Hangover-0.8.3 They were working on ARM and a bit of POWER9 for a while, before it had to be put on hold due to all the PE conversion work and resulting churn in WINE. Now it seems to have been restarted/rewritten.
    - mike_hearn 4y ago
      Ah! I used to work on Wine. Interesting project.

mwcampbell 4y ago

> without CPU portability, the latter seeming not that big of a deal these days anyway as cross compilers have got quite good and there are only two CPU archs in wide usage anyway Do we really want to surrender to the dominance of two proprietary ISAs? To me, the insistence on something like wasm bytecode is a kind of optimism, a hope that the current duopoly won't be permanent.

mike_hearn 4y ago
Well, Apple have proven several times now that you don't need a bytecode abstraction to allow CPU architecture transitions. They went from PowerPC to Intel to ARM whilst preserving the benefits of native code each time, using relatively simple stuff that stayed out of developers way and didn't require buyin to entirely new ways of coding or APIs. Rosetta 2 is just an ordinary AOT transpiler with a few magic flags in their ARM fork that lets them optionally match Intel semantics for code that needs it, other people should be able to match that. So bytecode is useful for lots of other reasons when paired with good runtimes, but it doesn't seem to be necessary to let you switch CPU ISAs or vendors. Meanwhile the costs are super high. WASM doesn't and isn't going to expose the real capabilities of the hardware any time soon, so it'll always lag behind in performance. The web has an unfortunate design goal of killing all the incentives OS and hardware makers have to add new features, because browser makers are these days pretty averse to allowing web pages to have special capabilities on specific underlying platforms, so features only get exposed to the web if every (major) OS and CPU combination supports them. But why add new features if apps will only start to use them if all your competitors also ship the same feature - there's no competitive advantage to be had there, which in turn means there's no point in investing in R&D. In turn that means that there's actually no point in designing or deploying a new ISA if everything is WASM. Why bother if it can't add any unique capabilities or nothing would use them? RISC-V says licensing is a good enough reason, but not many competitive design houses seem to care: making a high performance CPU is so expensive that ARM licensing costs just aren't a big deal in the grand scheme of things and in practice silicon OEMs seem fine with outsourcing ISA development to them. Leaning in to native code fixes the incentive problem, and software is constantly being updated these days anyway. If CPU portability was all that mattered it'd be cheaper and quicker to invest in better cross-compilation toolchains, like a standard file format for expressing all the URLs to all the different OS/CPU specific binaries for a program and build servers that could incrementally build and upload binaries for less popular ISAs in the background. I outline a proposal for an alternative web-like thing that works this way here (whilst also supporting wasm and other bytecodes): https://docs.google.com/document/d/1oDBw4fWyRNug3_f5mXWdlgDI4J5AbxVWKEeYr6hscT8/edit?usp=sharing https://docs.google.com/document/d/1oDBw4fWyRNug3_f5mXWdlgDI...
- 4y ago

evmar 4y ago

This (without a VM) is more or less what Wine does, but implementing the Windows ABI instead of the Linux one.

jakobson14 4y ago

It's astounding how few people know that this already exists, and has for years and years, in the from of user-mode QEMU. https://qemu-project.gitlab.io/qemu/user/index.html https://qemu-project.gitlab.io/qemu/user/index.html This is, by the way, how Hangover is supposed to work.

mike_hearn 4y ago
You're right, I didn't know about that. I thought QEMU is always a full OS emulator. Could you give a link for hangover? I'm not sure how to disambiguate in search from actual hangovers.
- jakobson14 4y ago
  https://github.com/AndreRH/hangover https://github.com/AndreRH/hangover https://www.phoronix.com/news/Wine-Hangover-2023-Open-Source https://www.phoronix.com/news/Wine-Hangover-2023-Open-Source https://www.phoronix.com/news/Hangover-0.8.3 https://www.phoronix.com/news/Hangover-0.8.3 They were working on ARM and a bit of POWER9 for a while, before it had to be put on hold due to all the PE conversion work and resulting churn in WINE. Now it seems to have been restarted/rewritten.
  - mike_hearn 4y ago
    Ah! I used to work on Wine. Interesting project.