4 ms·

Can anyone explain why the ARM ecosystem is like this? So many comments in this discussion about how all SBCs other than the RPi are terrible to get booted. Why

Can anyone explain why the ARM ecosystem is like this? So many comments in this discussion about how all SBCs other than the RPi are terrible to get booted. Why is it like that? Why hasn't the market settled on one bootchain? (Asking for my next home server!)

bri3d 4y ago
Because the way most ARM SoCs work is this: * Vendor integrates a bunch of IP (Cortex, codecs, GPU), tapes out SoC design. * Vendor has challenges hiring or retaining excellent software engineering team, because VLSI and hardware are the core competencies that make money, not software. * Vendor's struggling software engineering team hacks copy-pasted code together until they can produce a board support package against a single Linux kernel version, with no care to upstream. Once the device boots in a single configuration, they ship it. * Vendor's also struggling documentation team attempt to use sketchy descriptions from engineers to produce documentation. Often, internal interfaces are ignored or not documented since "it works" - only the external interfaces needed to make the chip run are documented. Now, an OEM buys the SoC and BSP. Sometimes they do some extra work to get the BSP to work with a newer kernel, and sometimes they don't. And if they do, they hack on top of the hacks, so the divergence increases and now there are multiple devicetree versions floating around (as happened with this Rock64 board).
- DCKing 4y ago
  I wonder at what point, if ever, the economics of ARM chip makers [1] starts to work in favor of ARM chips supporting full UEFI and ACPI for hardware initialization like x86 boxes do. I may be mistaken, but I thought some more "enterprise grade" ARM hardware already does UEFI+ACPI, as well as perhaps the Surface Pro X Windows 10 ARM device with a Qualcomm ARM SoC? You'd imagine at some point the maintenance costs for the chipmakers themselves as well as the increased interest / adoption rates from customers would tip the economics of all of this firmly towards standardization of this mess. Then again, I may not fully understand the economics here, or how much cheaper the current hacky approach is. [1]: Looking at you, Rockchip, Amlogic, Mediatek, Unisoc, Allwinner, Broadcom and the list goes on. I hope the Raspberry Pi Foundation + Broadcom can take the opportunity with the Raspberry Pi 5 or something to lead the way, but I'm not optimistic at all.
  - bri3d 4y ago
    Whether it's UEFI or U-Boot loading the DeviceTree blob doesn't really matter, no? A correct DeviceTree matching the drivers which ship with a given kernel version is still necessary. The big problems are when: * The DeviceTree is incorrect but works anyway because the peripheral drivers are ignoring it. * The DeviceTree is correct but the peripheral drivers had to be patched to work. UEFI doesn't seem to do much here, no? And even moving to ACPI doesn't really change anything - DeviceTree is just a simplified ACPI really, if the ACPI descriptors and what the peripheral drivers do don't match up, it still doesn't work.
    - DCKing 4y ago
      I suppose my comment is more about a culture shift rather than a technical suggestion, one supported by UEFI+ACPI :) Mainline Linux boots on pretty much all x86 hardware without needing explicit device trees, because the hardware is autodetected and initialized, and the hardware plays nice with the Linux driver (preferably mainline, but even that's not necessary). We could have that on ARM SBCs, if vendors were better. > And even moving to ACPI doesn't really change anything - DeviceTree is just a simplified ACPI really, if the ACPI descriptors and what the peripheral drivers do don't match up, it still doesn't work. I'm getting to the edges of my understanding of these things here, but wouldn't ACPI make it easier to support newer / mainline kernel versions without the vendor needing to supply a device tree specific to a kernel version they maintain? It would be nice if there was enough of a standard one could download a Debian ARM64 image from Debian's main website, and it would just boot and work on an ARM SBC. Even brand new x86 hardware can usually at least _boot_ and provide basic and useful I/O on Debian stable, and work pretty well on Debian sid.
      - rjsw 4y ago
        > It would be nice if there was enough of a standard one could download a Debian ARM64 image from Debian's main website, and it would just boot and work on an ARM SBC. NetBSD has one image that will boot on all supported ARM64 systems, it just uses the device tree or ACPI table supplied by the firmware to configure the hardware, there is nothing stopping Linux from doing the same.
- jandrese 4y ago
  Sometimes the above involves signing a NDA with companies like Broadcom so they can't release useful documentation without having a lawyer go over it first, and nobody wants to bother with that. Even worse, they can't release the code and have to release a giant poorly documented and buggy binary blob to support things like 3D acceleration and video decoding. And yes, fundamental features may or may not work, like resolutions other than 1920x1080 as described in the article.
3np 4y ago
https://tow-boot.org/ https://tow-boot.org/ is an initiative to improve on this.
fulafel 4y ago
It's the default state that happens with Linux ports. Unless there's an existing de facto standard OS whose boot interface can be piggybacked on. But the booting is just a small part of the mess. For the rest I guess the answer is similar - the default state of things is "not working" and a lot of things have to go right to make things run well. (How long did it take for RPi to get working hw video decoding OOTB with a distribution other than the board specific one from the SBC provider? Or does it even now?)

bri3d 4y ago

Because the way most ARM SoCs work is this: * Vendor integrates a bunch of IP (Cortex, codecs, GPU), tapes out SoC design. * Vendor has challenges hiring or retaining excellent software engineering team, because VLSI and hardware are the core competencies that make money, not software. * Vendor's struggling software engineering team hacks copy-pasted code together until they can produce a board support package against a single Linux kernel version, with no care to upstream. Once the device boots in a single configuration, they ship it. * Vendor's also struggling documentation team attempt to use sketchy descriptions from engineers to produce documentation. Often, internal interfaces are ignored or not documented since "it works" - only the external interfaces needed to make the chip run are documented. Now, an OEM buys the SoC and BSP. Sometimes they do some extra work to get the BSP to work with a newer kernel, and sometimes they don't. And if they do, they hack on top of the hacks, so the divergence increases and now there are multiple devicetree versions floating around (as happened with this Rock64 board).

DCKing 4y ago
I wonder at what point, if ever, the economics of ARM chip makers [1] starts to work in favor of ARM chips supporting full UEFI and ACPI for hardware initialization like x86 boxes do. I may be mistaken, but I thought some more "enterprise grade" ARM hardware already does UEFI+ACPI, as well as perhaps the Surface Pro X Windows 10 ARM device with a Qualcomm ARM SoC? You'd imagine at some point the maintenance costs for the chipmakers themselves as well as the increased interest / adoption rates from customers would tip the economics of all of this firmly towards standardization of this mess. Then again, I may not fully understand the economics here, or how much cheaper the current hacky approach is. [1]: Looking at you, Rockchip, Amlogic, Mediatek, Unisoc, Allwinner, Broadcom and the list goes on. I hope the Raspberry Pi Foundation + Broadcom can take the opportunity with the Raspberry Pi 5 or something to lead the way, but I'm not optimistic at all.
- bri3d 4y ago
  Whether it's UEFI or U-Boot loading the DeviceTree blob doesn't really matter, no? A correct DeviceTree matching the drivers which ship with a given kernel version is still necessary. The big problems are when: * The DeviceTree is incorrect but works anyway because the peripheral drivers are ignoring it. * The DeviceTree is correct but the peripheral drivers had to be patched to work. UEFI doesn't seem to do much here, no? And even moving to ACPI doesn't really change anything - DeviceTree is just a simplified ACPI really, if the ACPI descriptors and what the peripheral drivers do don't match up, it still doesn't work.
  - DCKing 4y ago
    I suppose my comment is more about a culture shift rather than a technical suggestion, one supported by UEFI+ACPI :) Mainline Linux boots on pretty much all x86 hardware without needing explicit device trees, because the hardware is autodetected and initialized, and the hardware plays nice with the Linux driver (preferably mainline, but even that's not necessary). We could have that on ARM SBCs, if vendors were better. > And even moving to ACPI doesn't really change anything - DeviceTree is just a simplified ACPI really, if the ACPI descriptors and what the peripheral drivers do don't match up, it still doesn't work. I'm getting to the edges of my understanding of these things here, but wouldn't ACPI make it easier to support newer / mainline kernel versions without the vendor needing to supply a device tree specific to a kernel version they maintain? It would be nice if there was enough of a standard one could download a Debian ARM64 image from Debian's main website, and it would just boot and work on an ARM SBC. Even brand new x86 hardware can usually at least _boot_ and provide basic and useful I/O on Debian stable, and work pretty well on Debian sid.
    - rjsw 4y ago
      > It would be nice if there was enough of a standard one could download a Debian ARM64 image from Debian's main website, and it would just boot and work on an ARM SBC. NetBSD has one image that will boot on all supported ARM64 systems, it just uses the device tree or ACPI table supplied by the firmware to configure the hardware, there is nothing stopping Linux from doing the same.
jandrese 4y ago
Sometimes the above involves signing a NDA with companies like Broadcom so they can't release useful documentation without having a lawyer go over it first, and nobody wants to bother with that. Even worse, they can't release the code and have to release a giant poorly documented and buggy binary blob to support things like 3D acceleration and video decoding. And yes, fundamental features may or may not work, like resolutions other than 1920x1080 as described in the article.

3np 4y ago

https://tow-boot.org/ https://tow-boot.org/ is an initiative to improve on this.

fulafel 4y ago

It's the default state that happens with Linux ports. Unless there's an existing de facto standard OS whose boot interface can be piggybacked on. But the booting is just a small part of the mess. For the rest I guess the answer is similar - the default state of things is "not working" and a lot of things have to go right to make things run well. (How long did it take for RPi to get working hw video decoding OOTB with a distribution other than the board specific one from the SBC provider? Or does it even now?)