4 ms·

ARM has a big problem …. compatibility. X86 absolutely destroys ARM in the software front. Give me an x86 cpu and I can run any version of linux on it with wid

ARM has a big problem …. compatibility. X86 absolutely destroys ARM in the software front. Give me an x86 cpu and I can run any version of linux on it with wide hardware support. Give me a random ARM computer, you’re lucky if you can get it to boot. You’re even luckier if the ARM CPU manufacturer releases documentation about how their chips work …. They prefer to keep inner workings secret.

capableweb 5y ago
> Give me a random ARM computer, you’re lucky if you can get it to boot. That's wildly inaccurate. Many distributions have ARM builds that works perfectly fine (at least for me, YMMV), have you tried any of them? Here is some of them: - Arch Linux - https://archlinuxarm.org/ https://archlinuxarm.org/ - Ubuntu - https://ubuntu.com/download/server/arm https://ubuntu.com/download/server/arm - Alpine Linux - https://alpinelinux.org/downloads/ https://alpinelinux.org/downloads/ - Elementary OS - https://blog.elementary.io/elementary-os-on-raspberry-pi/ https://blog.elementary.io/elementary-os-on-raspberry-pi/ - NixOS - https://nixos.wiki/wiki/NixOS_on_ARM https://nixos.wiki/wiki/NixOS_on_ARM
fork-bomber 5y ago
I don't think that's largely true anymore for server class hardware - which is the focus of the article. Arm came up with a bunch of standardisation requirements quite a while ago (see: https://developer.arm.com/architectures/system-architectures/arm-systemready https://developer.arm.com/architectures/system-architectures...) which have been quite successful especially for server designs. That was an absolute requirement in order for AArch64 to even be considered as an alternative in the datacenter space where it is now a very compelling alternative to x86_64. What I mean specifically is standardised support for firmware, hypervisor and operating system kernel interfaces for things like system bootstrap, power-perf control etc. Think ACPI, EFI, CPU capability discovery, DVFS, Idle management etc. Being unable to boot Linux on modern AArch64 server class hardware is actually increasingly rare thanks to the standardisation. Your comments are more applicable to the general Arm embedded systems scene where fragmentation is understandably rife. It was the price Arm had to pay to keep its royalty model in flight - "Pay the license fee, do what you will with the design".
3np 5y ago
The gap for generic arm64 has been closing a lot in recent years. These days IME the vast majority of fiddling doesn't have to do with the ARM SoC/CPU itself but rather with getting the right dtb (and sometimes firmware) for other integrated hardware, and u-boot. These issues aren't really inherent to the architecture per-se, more tied to the setups and practices for the kind of devices you tend to find available ARM SoCs/CPUs in. IME from dealing with embedded x86 devboards some years back it was a similar situation. I'd assume units marketed for server loads are comparable to x86 equivalents. I've been trying out Arch Linux on ARM on a secondary workstation recently. In most cases the only thing needed to get a non-supported package working is to add 'aarch64' to the list supported architectures in the PKGBUILD and then proceed like normal. > You’re even luckier if the ARM CPU manufacturer releases documentation about how their chips work …. They prefer to keep inner workings secret. This is the bigger practical issue. Rockchip are in general good, while the kind of chips you find in flagship phones require significant reverse-engineering are unapproachable for the non-hacker. But again, not so much "software-to-CPU", more "drivers-to-everything-around-the-CPU".
- floatboth 5y ago
  dtb is for embedded devices. Server/workstation class hardware uses ACPI.
  - 3np 5y ago
    Precisely, thanks for making it explicit.
- foxfluff 5y ago
  I wouldn't generalize Rockchip as good. You're lucky if they release one volume of their six volume reference manual, with "Confidential" plastered all over, and most peripherals completely undocumented beyond vague register names. If you're extra lucky, you'll find another leaked volume on a sketchy Chinese website. Anything else? NDA. And what about new chips? Look up datasheets for e.g. the (now popular) RK3566. You get a 58 page document which is just a general overview plus pinout and not much else. Does fuck all for you if you actually want to write drivers and get the thing to work.
maxwell86 5y ago
We bought a small ARM HPC cluster last summer, and everything worked out of the box. All our apps and dependencies just worked. Documentation is also excellent. ARM docs are really good, and go in much more detail than Intel and AMD docs about the inner working of their cores. That’s what we expected, since ARM licenses all their IP, and that’s what our vendor delivered. Everything we wanted to know about the Hw, there were docs and training materials ready for it.
- sidkshatriya 5y ago
  Is dealing with the slightly more relaxed memory/concurrency model of ARM been tricky or is it something you don't really encounter in practice?
  - maxwell86 5y ago
    We don’t write relaxed atomic kind of code directly (who does this, really?) We use MPI, pthreads primitives, c++ synchronization primitives, openmp, etc. These are portable and “just work”. Anecdotally, we haven’t run into any incorrect use of these in our apps yet that cause problems on ARM but not on x86 (although that would be a bug on both), but we aren’t doing anything super fancy.
- freemint 5y ago
  What scheduler do you use there?
  - maxwell86 5y ago
    slurm
floatboth 5y ago
> They prefer to keep inner workings secret https://github.com/tianocore/edk2-platforms/tree/master/Silicon/Ampere https://github.com/tianocore/edk2-platforms/tree/master/Sili... https://github.com/tianocore/edk2-platforms/tree/master/Silicon/Marvell https://github.com/tianocore/edk2-platforms/tree/master/Sili... https://github.com/tianocore/edk2-platforms/tree/master/Silicon/NXP https://github.com/tianocore/edk2-platforms/tree/master/Sili... https://github.com/tianocore/edk2-platforms/tree/master/Silicon/Hisilicon https://github.com/tianocore/edk2-platforms/tree/master/Sili...

capableweb 5y ago

> Give me a random ARM computer, you’re lucky if you can get it to boot. That's wildly inaccurate. Many distributions have ARM builds that works perfectly fine (at least for me, YMMV), have you tried any of them? Here is some of them: - Arch Linux - https://archlinuxarm.org/ https://archlinuxarm.org/ - Ubuntu - https://ubuntu.com/download/server/arm https://ubuntu.com/download/server/arm - Alpine Linux - https://alpinelinux.org/downloads/ https://alpinelinux.org/downloads/ - Elementary OS - https://blog.elementary.io/elementary-os-on-raspberry-pi/ https://blog.elementary.io/elementary-os-on-raspberry-pi/ - NixOS - https://nixos.wiki/wiki/NixOS_on_ARM https://nixos.wiki/wiki/NixOS_on_ARM

fork-bomber 5y ago

I don't think that's largely true anymore for server class hardware - which is the focus of the article. Arm came up with a bunch of standardisation requirements quite a while ago (see: https://developer.arm.com/architectures/system-architectures/arm-systemready https://developer.arm.com/architectures/system-architectures...) which have been quite successful especially for server designs. That was an absolute requirement in order for AArch64 to even be considered as an alternative in the datacenter space where it is now a very compelling alternative to x86_64. What I mean specifically is standardised support for firmware, hypervisor and operating system kernel interfaces for things like system bootstrap, power-perf control etc. Think ACPI, EFI, CPU capability discovery, DVFS, Idle management etc. Being unable to boot Linux on modern AArch64 server class hardware is actually increasingly rare thanks to the standardisation. Your comments are more applicable to the general Arm embedded systems scene where fragmentation is understandably rife. It was the price Arm had to pay to keep its royalty model in flight - "Pay the license fee, do what you will with the design".

3np 5y ago

The gap for generic arm64 has been closing a lot in recent years. These days IME the vast majority of fiddling doesn't have to do with the ARM SoC/CPU itself but rather with getting the right dtb (and sometimes firmware) for other integrated hardware, and u-boot. These issues aren't really inherent to the architecture per-se, more tied to the setups and practices for the kind of devices you tend to find available ARM SoCs/CPUs in. IME from dealing with embedded x86 devboards some years back it was a similar situation. I'd assume units marketed for server loads are comparable to x86 equivalents. I've been trying out Arch Linux on ARM on a secondary workstation recently. In most cases the only thing needed to get a non-supported package working is to add 'aarch64' to the list supported architectures in the PKGBUILD and then proceed like normal. > You’re even luckier if the ARM CPU manufacturer releases documentation about how their chips work …. They prefer to keep inner workings secret. This is the bigger practical issue. Rockchip are in general good, while the kind of chips you find in flagship phones require significant reverse-engineering are unapproachable for the non-hacker. But again, not so much "software-to-CPU", more "drivers-to-everything-around-the-CPU".

floatboth 5y ago
dtb is for embedded devices. Server/workstation class hardware uses ACPI.
- 3np 5y ago
  Precisely, thanks for making it explicit.
foxfluff 5y ago
I wouldn't generalize Rockchip as good. You're lucky if they release one volume of their six volume reference manual, with "Confidential" plastered all over, and most peripherals completely undocumented beyond vague register names. If you're extra lucky, you'll find another leaked volume on a sketchy Chinese website. Anything else? NDA. And what about new chips? Look up datasheets for e.g. the (now popular) RK3566. You get a 58 page document which is just a general overview plus pinout and not much else. Does fuck all for you if you actually want to write drivers and get the thing to work.

maxwell86 5y ago

We bought a small ARM HPC cluster last summer, and everything worked out of the box. All our apps and dependencies just worked. Documentation is also excellent. ARM docs are really good, and go in much more detail than Intel and AMD docs about the inner working of their cores. That’s what we expected, since ARM licenses all their IP, and that’s what our vendor delivered. Everything we wanted to know about the Hw, there were docs and training materials ready for it.

sidkshatriya 5y ago
Is dealing with the slightly more relaxed memory/concurrency model of ARM been tricky or is it something you don't really encounter in practice?
- maxwell86 5y ago
  We don’t write relaxed atomic kind of code directly (who does this, really?) We use MPI, pthreads primitives, c++ synchronization primitives, openmp, etc. These are portable and “just work”. Anecdotally, we haven’t run into any incorrect use of these in our apps yet that cause problems on ARM but not on x86 (although that would be a bug on both), but we aren’t doing anything super fancy.
freemint 5y ago
What scheduler do you use there?
- maxwell86 5y ago
  slurm

floatboth 5y ago

> They prefer to keep inner workings secret https://github.com/tianocore/edk2-platforms/tree/master/Silicon/Ampere https://github.com/tianocore/edk2-platforms/tree/master/Sili... https://github.com/tianocore/edk2-platforms/tree/master/Silicon/Marvell https://github.com/tianocore/edk2-platforms/tree/master/Sili... https://github.com/tianocore/edk2-platforms/tree/master/Silicon/NXP https://github.com/tianocore/edk2-platforms/tree/master/Sili... https://github.com/tianocore/edk2-platforms/tree/master/Silicon/Hisilicon https://github.com/tianocore/edk2-platforms/tree/master/Sili...