8 ms·

Huawei ExaGear: x86_64 apps on Arm 64-bit Linux systems

143 points by my123 6y ago

johndoe0815 6y ago
An interesting excerpt from the article: "ExaGear does not follow TSO completely by default, using heuristics instead." TSO emulation can be enabled fully, using heuristics, or turned off using a configuration option. It would be interesting to see benchmarks that analyze the performance (and correctness) impact of full TSO emulation vs. heuristics. Another interesting bit that is mentioned is that like Apple's M1, NVidia's Tegra K1 64-bit and Tegra Xavier support sequential consistency as the memory model.
- my123 6y ago
  Apple M1 has togglable TSO with ACTLR_EL1, not sequential consistency. You also don't run as TSO by default, has to be toggled on by the kernel when context switching to a given task.
- willxinc 6y ago
  This would make sense, as Eltech's engine was used extensively across multiple ports done by Nvidia for their Shield platform (RE5, MGRR)
paddim8 6y ago
This is really interesting. "ExaGear is a dynamic binary translation layer to run x86_32 and x86_64 applications on Arm 64-bit Linux systems." Couldn't this eventually be used in combination with Asahi Linux (project aiming to run Linux on Apple M1)? That would be incredible. Too bad it's from Huawei and doesn't seem to be open source though? I'm unable to find any additional information about this.
- StillBored 6y ago
  That is possible today with qemu/TCG. The assumption with exagear and some of these other JITs is that they are faster.
  - my123 6y ago
    And they are, Qemu isn't even remotely in the same performance category. (it was intended to be easily retargetable to other platforms instead of being tailored to them, not even using the hardware FPU at all for example)
    - stsquad 6y ago
      We do use the host FPU for a subset of floating point operations now. However it only really works for clean 32/64 IEEE FP so anything that goes through the x87 still needs software emulation.
      - my123 6y ago
        Wow, since which release did that finally happen? (I might also look at the other issues that I have with TCG code quality sometime) Might be worth it to update the wiki too.
        stsquad 6y ago
        Since v4.0.0 (see https://git.qemu.org/?p=qemu.git;a=commitdiff;h=a94b783952cc493cb241aabb1da8c7a830385baa https://git.qemu.org/?p=qemu.git;a=commitdiff;h=a94b783952cc...). We are always up for improving the code generation quality but of course there is a trade off with JITs given we are not compilers. I suspect there are still big wins if we can come up with a reasonable solution for re-generating a hot-path of basic blocks with much better optimisation.
        my123 6y ago
        Thank you. A tiered JIT will indeed cause much more optimisations to be made possible. I made a Qemu backend targeting LLVM before, but that turned out to be way too heavy to be usable, I wonder if it's worth revisiting that idea nowadays...
woodruffw 6y ago
This is the first I'm hearing of a dynamic binary translation tool from Huawei. This link only has a binary download and search results are scant; does anybody know any more about ExaGear?
- chmod775 6y ago
  See my comment here: https://news.ycombinator.com/item?id=25750402 https://news.ycombinator.com/item?id=25750402
- selfhoster11 6y ago
  ExaGear was originally developed by a company called Eltechs. I presume this must have been somehow acquired by Huawei.
- oefrha 6y ago
  Project page is here: https://www.huaweicloud.com/kunpeng/software/exagear.html https://www.huaweicloud.com/kunpeng/software/exagear.html Google (en-US) stupidly ranks it very low when searching for “Huawei exagear”, below some sketchy looking results, possibly due to language? Documentation is here https://support.huaweicloud.com/ug-exagear-kunpengdevps/kunpengexagear_06_0001.html https://support.huaweicloud.com/ug-exagear-kunpengdevps/kunp... and appear to be Chinese-only. Google Translate does a passable translation though.
obilgic 6y ago
Every time I travel abroad, I'm amazed by how much Huawei has penetrated every major market except U.S.
- coliveira 6y ago
  The reason is obvious: the US is enacting barriers to the internal market. But Huawei is developing all kinds of technology around the world.
  - heyflyguy 6y ago
    because spying.
    - throw48573 6y ago
      because "reports" of spying.
  - neither_color 6y ago
    The US is responding to asymmetrical access. Facebook, Google, Twitter, Instagram et al have been blocked for years in China. As of this week I'm ok with that but I digress. Apple's the only foreign cell phone maker that's really done alright in absolute dollar terms, but their market share is rather small. Access to the tantalizing "rising chinese middle class" is tantalizing and highly conditional. https://www.counterpointresearch.com/china-smartphone-share/ https://www.counterpointresearch.com/china-smartphone-share/
    - mytailorisrich 6y ago
      > Facebook, Google, Twitter, Instagram et al have been blocked for years in China. The Chinese government did not decide this on a whim, these companies are not willing to comply with Chinese rules within China. Whatever we might think of these rules the bottom line is that these companies are not in China because they have decided not to be. And when they try to move in, like Google did, their very employees argue against it. China is Apple's largest market, iirc, they are doing very well there. For smartphones their market share is 3 times their market share in India. Still, a lot of Chinese still cannot afford iPhones and China has many domestic manufacturers: Actual competition is fierce, this has nothing to do with politics. The argument that the US are retaliating against China blocking the FAANG is largely for the show but not reality.
FooBarWidget 6y ago
Any solutions for doing the reverse, namely running arm64 binaries on x86_64? Use case: building ARM software packages (like Docker containers) on x86_64 CI servers.
- Thev00d00 6y ago
  qemu?
  - my123 6y ago
    For something faster than Qemu, there's this: https://threedots.ovh/blog/2020/12/houdini-run-arm-32-bit-and-64-bit-applications-on-an-x86_64-system/ https://threedots.ovh/blog/2020/12/houdini-run-arm-32-bit-an...
  - xuhu 6y ago
    BTW, how does ExaGear compare to qemu for running x86_64 on arm64 hosts ?
  - wyldfire 6y ago
    QEMU's sysemu is a little unwieldy for CI, but the linux userspace support is perfect for this use case. In case you haven't encountered it -- it's terribly clever. Userspace instructions are translated just like they are in full system mode, but when a system call is encountered, the arguments are translated between the host and guest system. So the guest architecture linux program 'just works'. There's also hooks you can enable in binfmt-misc to make the loader invoke the qemu binary automagically.
    - codetrotter 6y ago
      Sounds interesting. Do you have a link with more info about these things?
      - wyldfire 6y ago
        [1] [2] are handy links. [1] https://www.qemu.org/docs/master/user/main.html https://www.qemu.org/docs/master/user/main.html [2] https://wiki.debian.org/QemuUserEmulation https://wiki.debian.org/QemuUserEmulation
chmod775 6y ago
Note that Eltech's ExaGear used to be a thing, and has now largely disappeared, along with the company homepage. I don't know whether they've been bought by Huawei, who is now reviving the product under their own name. Original announcement of discontinuation: --- Eltechs Weekly News Digest --- We have discontinued ExaGear... Dear ExaGear users! We're really sorry to announce our business decision about discontinuing our services and products. You'll be able to continue using all of your purchased products, the license checking functionality will work. You can download our most recent ExaGear Desktop version 3.1 below: ExaGear Desktop for Raspberry Pi 3 ExaGear Desktop for Raspberry Pi 2 ExaGear Desktop for Raspberry Pi 1/Zero ExaGear Desktop for Odroid XU4 ExaGear Desktop for Odroid C2 ExaGear Desktop for Odroid 32bit ExaGear Desktop for NVIDIA Jetson TX1 / TX2 ExaGear Desktop for NVIDIA Jetson TK1 ExaGear Desktop for CubieBoard ExaGear Desktop for Banana Pi ExaGear Desktop for ARMv7 ExaGear Desktop for ARMv8 We strongly recommend to save ExaGear installation package on your own hard drive because the download won't be available after February 28th, 2019. Source: https://web.archive.org/web/20190227095445/http://forum.eltechs.com/viewtopic.php?f=4&t=1609 https://web.archive.org/web/20190227095445/http://forum.elte...
- my123 6y ago
  Eltechs' ExaGear at the time only provided x86_32 compatibility on top of 32-bit Arm. This is a major departure from that, my best guess would be that Huawei purchased the company and then poured lots of resources to it.
  - chmod775 6y ago
    Given the timeline Eltech had likely been shut down for at least a year. Then Apple came around with their x86 to ARM translation layer, prompting Huawei to go shopping for a competitor, seeing the defunct Eltech, and buying their IP. Sounds like a late happy ending for a failed company that was just a few years too early with their product. I hope the founders got a good deal.
    - sangnoir 6y ago
selfhoster11 6y ago
Anybody know why this is showing up with Huawei branding after Eltechs shut down? I presume this must be because ExaGear was acquired, but I cannot find any information online about this, at least in English.
throwaway4good 6y ago
I wonder if this is related to Huawei's effort to bring forth desktop PCs and laptops based on their Kirin/Kunpeng ARM CPUs? Running a desktop version of their HarmonyOS with x86 apps in a compability layer.
trizic 6y ago
Also check out Box86: https://github.com/ptitSeb/box86 https://github.com/ptitSeb/box86 Impressive to see people playing Half-Life 2, Unreal Tournament 99 and 2004 on a Raspberry Pi 4. Curious if this could be extended to Emscripten (JIT for WebAssembly/Emscripten?) which would be perfect for archiving and portability. Closest project I have seen is DOSBox-X which seems to already have an Emscripten port: https://github.com/joncampbell123/dosbox-x https://github.com/joncampbell123/dosbox-x HL2: https://www.youtube.com/watch?v=Lq5dJ0oc_6k https://www.youtube.com/watch?v=Lq5dJ0oc_6k UT2004: https://www.youtube.com/watch?v=4VOAeOQcrdo https://www.youtube.com/watch?v=4VOAeOQcrdo
- spijdar 6y ago
  Another fun thing about box86 is it isn't specifically limited to ARM. Its JIT translator is ARM-specific, but it includes a pure emulator which will work on any CPU. There's additional work to be done for each port, but I got it working on 32 bit PowerPC LE, allowing one to run x86 apps on IBM's POWER processors. For all the few dozens of people this may be relevant to :-)
- ekianjo 6y ago
  More info here about box86 https://www.giantpockets.com/box86-run-x86-code-and-games-on-arm/ https://www.giantpockets.com/box86-run-x86-code-and-games-on...
- dleslie 6y ago
  Sadly, box86 doesn't support aarch64.
  - Jkvngt 6y ago
    Does Raspbian yet?
    - kelnos 6y ago
      Somewhat, they have a beta aarch64 image: https://downloads.raspberrypi.org/raspios_arm64/images/ https://downloads.raspberrypi.org/raspios_arm64/images/
Klasiaster 6y ago
What qemu-user makes so special for me is that is can be pre-loaded by the kernel, allowing any foreign arch chroot to directly work, nice for containers¹. I've not read if this here does support it, too. ¹ Once `qemu-user-static` is installed on your system, try `docker run --rm -ti docker.io/arm64v8/alpine` on amd64 or vice versa
- my123 6y ago
  This supports it too. See the ExaGear4Docker folder in the install archive.
- rfoo 6y ago
  That's just a general kernel feature named binfmt_misc [1], just register the interpreter with the O flag. You can write your own binary translator and have the kernel run it for you, too. [1] https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html https://www.kernel.org/doc/html/latest/admin-guide/binfmt-mi...
danielEM 6y ago
Does it really matter for Linux world where you jest compile software from open sources anyway?
- HideousKojima 6y ago
  There's plenty of software where the source isn't available, and plenty of open source software where it won't compile for a different architecture without significant work because of runtimes or other dependencies
pabs3 6y ago
I wonder if Huawei plan to open source this? That they make it publicly available makes it seem like they don't plan to monetise it, so the next step would be to open source it, unless patents or third-party code prevent that.
nunna 6y ago
how to get this new EXAHEAR VERSION located here? https://www.huaweicloud.com/kunpeng/software/exagear.html https://www.huaweicloud.com/kunpeng/software/exagear.html i TRYED to contact huawei france support but they asked me to call China support. Tryed to create a china account but it require a China phone line number. Then how to get it?

143 points by my123 6y ago

An interesting excerpt from the article: "ExaGear does not follow TSO completely by default, using heuristics instead." TSO emulation can be enabled fully, using heuristics, or turned off using a configuration option. It would be interesting to see benchmarks that analyze the performance (and correctness) impact of full TSO emulation vs. heuristics. Another interesting bit that is mentioned is that like Apple's M1, NVidia's Tegra K1 64-bit and Tegra Xavier support sequential consistency as the memory model.

my123 6y ago
Apple M1 has togglable TSO with ACTLR_EL1, not sequential consistency. You also don't run as TSO by default, has to be toggled on by the kernel when context switching to a given task.
willxinc 6y ago
This would make sense, as Eltech's engine was used extensively across multiple ports done by Nvidia for their Shield platform (RE5, MGRR)

paddim8 6y ago

This is really interesting. "ExaGear is a dynamic binary translation layer to run x86_32 and x86_64 applications on Arm 64-bit Linux systems." Couldn't this eventually be used in combination with Asahi Linux (project aiming to run Linux on Apple M1)? That would be incredible. Too bad it's from Huawei and doesn't seem to be open source though? I'm unable to find any additional information about this.

StillBored 6y ago
That is possible today with qemu/TCG. The assumption with exagear and some of these other JITs is that they are faster.
- my123 6y ago
  And they are, Qemu isn't even remotely in the same performance category. (it was intended to be easily retargetable to other platforms instead of being tailored to them, not even using the hardware FPU at all for example)
  - stsquad 6y ago
    We do use the host FPU for a subset of floating point operations now. However it only really works for clean 32/64 IEEE FP so anything that goes through the x87 still needs software emulation.
    - my123 6y ago
      Wow, since which release did that finally happen? (I might also look at the other issues that I have with TCG code quality sometime) Might be worth it to update the wiki too.
      - stsquad 6y ago
        Since v4.0.0 (see https://git.qemu.org/?p=qemu.git;a=commitdiff;h=a94b783952cc493cb241aabb1da8c7a830385baa https://git.qemu.org/?p=qemu.git;a=commitdiff;h=a94b783952cc...). We are always up for improving the code generation quality but of course there is a trade off with JITs given we are not compilers. I suspect there are still big wins if we can come up with a reasonable solution for re-generating a hot-path of basic blocks with much better optimisation.
        my123 6y ago
        Thank you. A tiered JIT will indeed cause much more optimisations to be made possible. I made a Qemu backend targeting LLVM before, but that turned out to be way too heavy to be usable, I wonder if it's worth revisiting that idea nowadays...

woodruffw 6y ago

This is the first I'm hearing of a dynamic binary translation tool from Huawei. This link only has a binary download and search results are scant; does anybody know any more about ExaGear?

chmod775 6y ago
See my comment here: https://news.ycombinator.com/item?id=25750402 https://news.ycombinator.com/item?id=25750402
selfhoster11 6y ago
ExaGear was originally developed by a company called Eltechs. I presume this must have been somehow acquired by Huawei.
oefrha 6y ago
Project page is here: https://www.huaweicloud.com/kunpeng/software/exagear.html https://www.huaweicloud.com/kunpeng/software/exagear.html Google (en-US) stupidly ranks it very low when searching for “Huawei exagear”, below some sketchy looking results, possibly due to language? Documentation is here https://support.huaweicloud.com/ug-exagear-kunpengdevps/kunpengexagear_06_0001.html https://support.huaweicloud.com/ug-exagear-kunpengdevps/kunp... and appear to be Chinese-only. Google Translate does a passable translation though.

obilgic 6y ago

Every time I travel abroad, I'm amazed by how much Huawei has penetrated every major market except U.S.

coliveira 6y ago
The reason is obvious: the US is enacting barriers to the internal market. But Huawei is developing all kinds of technology around the world.
- heyflyguy 6y ago
  because spying.
  - throw48573 6y ago
    because "reports" of spying.
- neither_color 6y ago
  The US is responding to asymmetrical access. Facebook, Google, Twitter, Instagram et al have been blocked for years in China. As of this week I'm ok with that but I digress. Apple's the only foreign cell phone maker that's really done alright in absolute dollar terms, but their market share is rather small. Access to the tantalizing "rising chinese middle class" is tantalizing and highly conditional. https://www.counterpointresearch.com/china-smartphone-share/ https://www.counterpointresearch.com/china-smartphone-share/
  - mytailorisrich 6y ago
    > Facebook, Google, Twitter, Instagram et al have been blocked for years in China. The Chinese government did not decide this on a whim, these companies are not willing to comply with Chinese rules within China. Whatever we might think of these rules the bottom line is that these companies are not in China because they have decided not to be. And when they try to move in, like Google did, their very employees argue against it. China is Apple's largest market, iirc, they are doing very well there. For smartphones their market share is 3 times their market share in India. Still, a lot of Chinese still cannot afford iPhones and China has many domestic manufacturers: Actual competition is fierce, this has nothing to do with politics. The argument that the US are retaliating against China blocking the FAANG is largely for the show but not reality.

FooBarWidget 6y ago

Any solutions for doing the reverse, namely running arm64 binaries on x86_64? Use case: building ARM software packages (like Docker containers) on x86_64 CI servers.

Thev00d00 6y ago
qemu?
- my123 6y ago
  For something faster than Qemu, there's this: https://threedots.ovh/blog/2020/12/houdini-run-arm-32-bit-and-64-bit-applications-on-an-x86_64-system/ https://threedots.ovh/blog/2020/12/houdini-run-arm-32-bit-an...
- xuhu 6y ago
  BTW, how does ExaGear compare to qemu for running x86_64 on arm64 hosts ?
- wyldfire 6y ago
  QEMU's sysemu is a little unwieldy for CI, but the linux userspace support is perfect for this use case. In case you haven't encountered it -- it's terribly clever. Userspace instructions are translated just like they are in full system mode, but when a system call is encountered, the arguments are translated between the host and guest system. So the guest architecture linux program 'just works'. There's also hooks you can enable in binfmt-misc to make the loader invoke the qemu binary automagically.
  - codetrotter 6y ago
    Sounds interesting. Do you have a link with more info about these things?
    - wyldfire 6y ago
      [1] [2] are handy links. [1] https://www.qemu.org/docs/master/user/main.html https://www.qemu.org/docs/master/user/main.html [2] https://wiki.debian.org/QemuUserEmulation https://wiki.debian.org/QemuUserEmulation

chmod775 6y ago

Note that Eltech's ExaGear used to be a thing, and has now largely disappeared, along with the company homepage. I don't know whether they've been bought by Huawei, who is now reviving the product under their own name. Original announcement of discontinuation: --- Eltechs Weekly News Digest --- We have discontinued ExaGear... Dear ExaGear users! We're really sorry to announce our business decision about discontinuing our services and products. You'll be able to continue using all of your purchased products, the license checking functionality will work. You can download our most recent ExaGear Desktop version 3.1 below: ExaGear Desktop for Raspberry Pi 3 ExaGear Desktop for Raspberry Pi 2 ExaGear Desktop for Raspberry Pi 1/Zero ExaGear Desktop for Odroid XU4 ExaGear Desktop for Odroid C2 ExaGear Desktop for Odroid 32bit ExaGear Desktop for NVIDIA Jetson TX1 / TX2 ExaGear Desktop for NVIDIA Jetson TK1 ExaGear Desktop for CubieBoard ExaGear Desktop for Banana Pi ExaGear Desktop for ARMv7 ExaGear Desktop for ARMv8 We strongly recommend to save ExaGear installation package on your own hard drive because the download won't be available after February 28th, 2019. Source: https://web.archive.org/web/20190227095445/http://forum.eltechs.com/viewtopic.php?f=4&t=1609 https://web.archive.org/web/20190227095445/http://forum.elte...

my123 6y ago
Eltechs' ExaGear at the time only provided x86_32 compatibility on top of 32-bit Arm. This is a major departure from that, my best guess would be that Huawei purchased the company and then poured lots of resources to it.
- chmod775 6y ago
  Given the timeline Eltech had likely been shut down for at least a year. Then Apple came around with their x86 to ARM translation layer, prompting Huawei to go shopping for a competitor, seeing the defunct Eltech, and buying their IP. Sounds like a late happy ending for a failed company that was just a few years too early with their product. I hope the founders got a good deal.
  - sangnoir 6y ago

selfhoster11 6y ago

Anybody know why this is showing up with Huawei branding after Eltechs shut down? I presume this must be because ExaGear was acquired, but I cannot find any information online about this, at least in English.

throwaway4good 6y ago

I wonder if this is related to Huawei's effort to bring forth desktop PCs and laptops based on their Kirin/Kunpeng ARM CPUs? Running a desktop version of their HarmonyOS with x86 apps in a compability layer.

trizic 6y ago

Also check out Box86: https://github.com/ptitSeb/box86 https://github.com/ptitSeb/box86 Impressive to see people playing Half-Life 2, Unreal Tournament 99 and 2004 on a Raspberry Pi 4. Curious if this could be extended to Emscripten (JIT for WebAssembly/Emscripten?) which would be perfect for archiving and portability. Closest project I have seen is DOSBox-X which seems to already have an Emscripten port: https://github.com/joncampbell123/dosbox-x https://github.com/joncampbell123/dosbox-x HL2: https://www.youtube.com/watch?v=Lq5dJ0oc_6k https://www.youtube.com/watch?v=Lq5dJ0oc_6k UT2004: https://www.youtube.com/watch?v=4VOAeOQcrdo https://www.youtube.com/watch?v=4VOAeOQcrdo

spijdar 6y ago
Another fun thing about box86 is it isn't specifically limited to ARM. Its JIT translator is ARM-specific, but it includes a pure emulator which will work on any CPU. There's additional work to be done for each port, but I got it working on 32 bit PowerPC LE, allowing one to run x86 apps on IBM's POWER processors. For all the few dozens of people this may be relevant to :-)
ekianjo 6y ago
More info here about box86 https://www.giantpockets.com/box86-run-x86-code-and-games-on-arm/ https://www.giantpockets.com/box86-run-x86-code-and-games-on...
dleslie 6y ago
Sadly, box86 doesn't support aarch64.
- Jkvngt 6y ago
  Does Raspbian yet?
  - kelnos 6y ago
    Somewhat, they have a beta aarch64 image: https://downloads.raspberrypi.org/raspios_arm64/images/ https://downloads.raspberrypi.org/raspios_arm64/images/

Klasiaster 6y ago

What qemu-user makes so special for me is that is can be pre-loaded by the kernel, allowing any foreign arch chroot to directly work, nice for containers¹. I've not read if this here does support it, too. ¹ Once `qemu-user-static` is installed on your system, try `docker run --rm -ti docker.io/arm64v8/alpine` on amd64 or vice versa

my123 6y ago
This supports it too. See the ExaGear4Docker folder in the install archive.
rfoo 6y ago
That's just a general kernel feature named binfmt_misc [1], just register the interpreter with the O flag. You can write your own binary translator and have the kernel run it for you, too. [1] https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html https://www.kernel.org/doc/html/latest/admin-guide/binfmt-mi...

danielEM 6y ago

Does it really matter for Linux world where you jest compile software from open sources anyway?

HideousKojima 6y ago
There's plenty of software where the source isn't available, and plenty of open source software where it won't compile for a different architecture without significant work because of runtimes or other dependencies

pabs3 6y ago

I wonder if Huawei plan to open source this? That they make it publicly available makes it seem like they don't plan to monetise it, so the next step would be to open source it, unless patents or third-party code prevent that.

nunna 6y ago

how to get this new EXAHEAR VERSION located here? https://www.huaweicloud.com/kunpeng/software/exagear.html https://www.huaweicloud.com/kunpeng/software/exagear.html i TRYED to contact huawei france support but they asked me to call China support. Tryed to create a china account but it require a China phone line number. Then how to get it?