12 ms·

Clang now makes binaries an original Pi B+ can't run

375 points by FPGAhacker 3y ago

frizlab 3y ago
Title is misleading
- eimrine 3y ago
  Because "as a default" statement is missing?
  - jenadine 3y ago
    Yes. It now sounds like it is completely broken. But you can just fix it with a flag. And the change of default was probably an unintentional bug.
- usr1106 3y ago
  Aka called clickbait. Although the bug and the workaround are useful to know for everyone working with that machine.
- blahgeek 3y ago
  Yes. It should be “clang does not correctly detect host architecture in raspberry pi B+”
johnklos 3y ago
You'll see a whole bandwagon of people saying things like, "supporting old hardware is BAD! It takes time and money that nobody has!", as though someone needs to be hired to sit around and do nothing but pore over code and constantly rewrite code for old hardware. There's plenty of evidence to the contrary, but since when has evidence mattered when it comes to defending the right of big business / big distro to do whatever they want? ;) Really, this is just laziness and sloppiness on the Linux distro makers' part. Any amount of testing would catch this. Thanks, Rachel!
- Elucalidavah 3y ago
  > sit around and do nothing but pore over code and constantly rewrite code for old hardware In case of refactoring / restructuring, that's exactly so. But "drop support for this old hardware" is meant to be an intended decision with a clear deprecation warning, not accidentally.
  - atemerev 3y ago
    The bazaar doesn’t work like this. There are no incentives to support old hardware. If there are enough people with old hardware, their activity might be enough to do something. Otherwise — puff, gone. Open source is already a miracle. The fact that something works somewhere is a miracle. I don’t tempt the powers and I don’t demand even more miracles to satisfy some perfectionist urges.
    - deleted 3y ago
      [deleted]
- wmf 3y ago
  I think RPi has struck a reasonable balance where the mainstream Linux community doesn't really support ancient ARMv6 so RPI themselves maintains forked software (e.g. Raspbian). This way the cost of legacy is borne by those who benefit from it, not everyone.
  - deaddodo 3y ago
    Raspbian existed well before ARMv6 support dropped off. It's been their main distro from the outset, but mainstream distros with ARM builds only removed support for ARMv6 in the last 1-3 years (depending on distro).
    - account42 3y ago
      Pretty sure Debian's armhf images have always required ARMv7 or at least for much longer than 3 years. There are also armel images but those don't use hardware floats which makes them much less performant than what the original Pi is capable of. Pretty sure that that mismatch is why raspian exists in the first place.
teddyh 3y ago
It would probably be helpful to know the output of the command “dpkg-architecture” and the contents of the file “/etc/os-release”. Otherwise it will be hard to make any useful comments.
- deleted 3y ago
  [deleted]
- cbmuser 3y ago
  Exactly. The article omits information that is fundamental to being able to fix this problem.
ninepoints 3y ago
[flagged]
- ajb 3y ago
  It's not explicitly stated but in her reproduction she is actually running it on the pi, and compilers very much do normally compile for the arch that they are running on by default. A compiler which compiles for a different arch is called a cross-compiler. Unless you explicitly asked for a cross compile, it is indeed surprising for a compiler to emit a binary that won't run on the same machine.
  - P-Nuts 3y ago
    It’s explicitly stated: “I figured, okay, maybe it's doing some optimization because it was built on the 4B. So, I went and did a build on the B+ natively. It also broke.” So she was originally compiling on a newer RPi, but immediately switched to compiling on the same one that couldn’t run it. And I believe the actual system images for RPi are built to support all models (there aren’t separate downloads for different models unless you want 64-bit) so it’s not crazy to think that everything would work building on a newer model. This is just a packaging defect, Clang/LLVM isn’t being configured correctly by RPiOS (Raspbian).
  - thaumasiotes 3y ago
    > It's not explicitly stated but in her reproduction she is actually running it on the pi This is what the post says: >> This used to work in the old version (bullseye). It now breaks in the current one (bookworm). I figured, okay, maybe it's doing some optimization because it was built on the 4B. So, I went and did a build on the B+ natively. It also broke. If that's not an explicit statement that she's compiling on the target platform, what is?
- wmf 3y ago
  It sounds like clang running on the RPi 1 generates code that doesn't run. Usually compilers default to targeting whatever ISA it's running on but that doesn't seem to be the case here.
schemescape 3y ago
I didn't see it addressed here or in the article: this is a bug, right? Edit: oddly, after searching LLVM bugs, I found a bug that sounds pretty much exactly like this issue... but it's from 2012 and is closed (although the final couple of comments make it sound like maybe it wasn't actually fixed--note: I only skimmed the comments and I probably misunderstood): https://github.com/llvm/llvm-project/issues/13989 https://github.com/llvm/llvm-project/issues/13989 Edit again: I forgot about the comment at the end of the article that clarifies that explicitly passing the target results in a working program. In that case, it sounds like some sort of configuration bug--I would assume (but am not certain) that the default target would be the current processor, at least on Unix. That bug I linked was probably about producing incorrect code even when the target was set correctly which, thankfully, isn't happening today.
- deleted 3y ago
  [deleted]
- wmf 3y ago
  Yeah, this is not the behavior people expect.
  - NikkiA 3y ago
    I would expect a arm64 machine to not build a arm32 compatible binary by default, it's the same as running clang on a x86-64 host and expecting it to produce 386 compatible binaries without a -march=i386 somewhere. The weird clang install on a fresh B+ install is more puzzling, unless there's some user error somewhere.
- deleted 3y ago
  [deleted]
- Arnavion 3y ago
  Yes, your bug is about the compiler emitting armv7 instructions despite being told to target armv6. Rachel fixed her problem by telling the compiler to target armv6. So I assume your bug is indeed already fixed and not related to Rachel's problem.
- 3y ago
deleted 3y ago
[deleted]
stephen_g 3y ago
Wow, looking at the history of the ARM generation the original versions of the Raspberry Pi uses, it’s hard to believe it’s so old! When the Raspberry Pi B+ was released (2014), the ARM core it used was already 11 years old (using the ARM1176 core from 2003). So it’s not unbelievable that you might need to start supplying an arch flag to produce compatible code building on a different platform (like the newer Raspberry Pi the article says they first built on). As others have said, it does seem like a misconfiguration (perhaps in the defaults shipped by their distribution) that the correct arch is not picked by default when building on the Raspberry Pi B+ itself.
- yjftsjthsd-h 3y ago
  > When the Raspberry Pi B+ was released (2014), the ARM core it used was already 11 years old (using the ARM1176 core from 2003). IIRC the original Pi used leftover chips from a TV box, which is the kind of product that IME never ships more compute than they have to, for price reasons.
  - einr 3y ago
    TV boxes IME usually ship with less compute than they have to [in order to provide reasonable UX] ;)
    - yjftsjthsd-h 3y ago
      Eh, the result may suck, but I don't think it's usually a hardware problem.
    - adhesive_wombat 3y ago
      Anything to satisfy whichever law it is that says "lagginess remains constant".
      - actionfromafar 3y ago
        "Huh, this CPU is kind of snappy running native code now. What should we do?" "Let's move application development to Python then, I suppose." "Thanks, that fixed it." Probably what happened in my 55" smart TV dev team.
dang_rachel 3y ago
[dead]
dang_rachel2 3y ago
[dead]
matja 3y ago
clang/clang++ read from /etc/env.d/gcc to get the target flags/profile, it's up to the OS to maintain them and make sure they're correct, looks like that didn't happen for this OS. My Gentoo ARM SBC based on an even more ancient armv4 arch has been chugging along just fine with the latest gcc/clang updates: grep CTARGET /etc/env.d/gcc -r /etc/env.d/gcc/armv4tl-softfloat-linux-gnueabi-11.3.0:CTARGET="armv4tl-softfloat-linux-gnueabi"
- contingencies 3y ago
  Gentoo always works .. it just takes longer :)
- Arnavion 3y ago
  /etc/env.d is a Gentoo-specific directory to define default env vars for user sessions. It's not a feature of clang to read that directory, so it's not correct to assume other distros would have it. It's just that Gentoo's compiler setup reads the CTARGET env var to select the target, and Gentoo uses /etc/env.d to set it.
  - matja 3y ago
    Is that why other distros break? :)
auselen 3y ago
Confused article? You make a host/native build instead of cross and expect it to work on some other machine?
- daviddoran 3y ago
  No. Half way through the article she specifically starts doing everything on the B+ (the old RPI with the issue).
  - auselen 3y ago
    Thanks for that. I didn’t notice she switch to B+ later.
deleted 3y ago
[deleted]
opello 3y ago
Seems like the problem is likely a configuration target change in the clang-13 package that's current for bookworm. Specifically because under bullseye (and clang-11) the default target is armv6k-unknown-linux-gnueabihf while under bookworm (and clang-13) the default target is arm-unknown-linux-gnueabihf. Or maybe the default changed for the given build configuration on the LLVM side?
- opello 3y ago
  I really wish I understood the Debian change management process better. I guess I don't even really know if Raspbian is actually maintained by Debian. But, when comparing [1] to [2], the rules file has a nice test that says "if DEB_HOST_ARCH is armhf, set the LLVM_HOST_TRIPLE to armv6k..." which seems to confirm a build configuration change. [1] http://raspbian.raspberrypi.org/raspbian/pool/main/l/llvm-toolchain-11/llvm-toolchain-11_11.0.1-2~deb10u1+rpi1.debian.tar.xz http://raspbian.raspberrypi.org/raspbian/pool/main/l/llvm-to... [2] http://raspbian.raspberrypi.org/raspbian/pool/main/l/llvm-toolchain-13/llvm-toolchain-13_13.0.1-6~deb10u4+rpi1.debian.tar.xz http://raspbian.raspberrypi.org/raspbian/pool/main/l/llvm-to...
  - orra 3y ago
    To answer your incidental question, Raspian is maintained by Raspberry Pi folks, not Debian.
    - opello 3y ago
      Ah, thanks! I dug a little more and found that the original test [1] from llvm-toolchain was a little different in the upstream Debian repository. It set the triple to armv7l for armhf hosts. But I haven't yet found a similar repository on the Raspbian side. I guess I'd expect to find it within their GitHub org, but my searching didn't reveal it. [1] https://salsa.debian.org/pkg-llvm-team/llvm-toolchain/-/commit/af56187b7d04d718f3f2229691885b102bb026a5 https://salsa.debian.org/pkg-llvm-team/llvm-toolchain/-/comm...
      - 3y ago
JonChesterfield 3y ago
I doubt this is a deliberate change. Picking up information from the environment - a sibling mentions /etc/env.d/gcc - seems fairly likely. I'd guess the default triple is something like arm-unknown-linux unless clang finds or is is told something more specific to use, and the mechanism by which it gets told to use something more specific has fallen over. This might mean there are no arm v6 buildbots running, or it might mean there are ones running but the implicit configuration is still working on them. LLVM is a really good cross compiler. Build for any target from any target, no trouble. Clang is less compelling - if it's built with the target, and you manage to tell it what target to build for, it'll probably do the right thing (as in this post - it guessed wrong, but given more information, did the right thing). Then the runtime library story is worse again - you've built for armv4 or whatever, but now you need to find a libc etc for it, and you might need to tell the compiler where those libraries and headers are, and for that part I'm still unclear on the details.
- rcarmo 3y ago
  Most distros and compilers effectively dropped ARMv6 a couple of years back - I had similar trouble building binaries for my old Synology NAS.
  - anthk 3y ago
    Alpine Linux might still support it I think.
- hulitu 3y ago
  > Picking up information from the environment - a sibling mentions /etc/env.d/gcc - seems fairly likely. Why would CLANG do this ?
  - elteto 3y ago
    Clang already replicates a bunch of flags, macros, and behaviors from gcc. The objective is to be a drop-in replacement, and make the developer experience much nicer when migrating. There are some rough corners, of course, but overall it’s actually very nice.
fsniper 3y ago
Title is unfortunately sensational. This is a default target change. Turns out clang still can build binaries for the Pi B+. You just need to be explicit about the architecture. So perhaps a small title change that's more clear about this being only default setting change?
- nottorp 3y ago
  Doesn’t seem so sensational when it can’t build binaries for the target machine… on the target machine itself…
  - cbmuser 3y ago
    It’s sensational because it’s wrong. LLVM still supports even ARMv5T which is the baseline of Debian’s armel port.
  - dagmx 3y ago
    It can, it just doesn’t by default. Which is what the person you’re replying to is saying.
    - nottorp 3y ago
      And... does it make sense to you... when you're not cross compiling?
      - dagmx 3y ago
        The point is that it objectively CAN compile to the right target. The capability is not broken. It however DOESNT due to a configuration bug. Therefore it doesn’t have to make sense because it’s clearly not intentional. your sentence saying “it can’t build” is therefore incorrect. It’s the distinction between the two capitalized words above.
        brian-armstrong 3y ago
        If "clang helloworld.c" doesn't produce a working a.out out of the box, I think it's fair to say builds are broken. Plenty of projects won't build in those circumstances without some assistance.
draw_down 3y ago
[dead]
cbmuser 3y ago
The article doesn’t mention whether Debian or Raspian was installed. And, in case of Debian, whether the armel or armhf port is being used. Without that information, it’s pretty pointless to make claims about the instruction set LLVM compiles to because that’s a matter of what native target LLVM has been configured for. FWIW, in Debian, llvm-toolchaim-snapshot still supports armel which uses ARMv5T as the baseline (there is currently an unrelated bug in LLVM’s OpenMP library though which prevents a successful build).
- jchw 3y ago
  What's weird is that the Clang binary is clearly compiled for an instruction set that is compatible with the Pi B+, but it doesn't target an instruction set that is compatible with the Pi B+. This is genuinely weird, since that's not meant to be a cross-compiler; in theory, the host and the target should be the same. Presumably the image is Raspbian. I don't see a reason why not to assume that.
  - guipsp 3y ago
    One key thing is missing from your comment (which explains the 'weirdness') - clang, and other llvm-family tools are cross-compatible by default. There is no separate cross compilation binary. This is just a configuration bug.
    - jchw 3y ago
      Yes, that's true, although it doesn't actually explain the weirdness. I compile Clang all the time and there's no obvious reason why you'd get cross-compiled binaries out of Clang if you just compile and install it normally. The bug, configuration or otherwise, is the weirdness.
rschu1ze 3y ago
The database I work on (ClickHouse) tries hard to stay compatible with really old hardware. The standard ARM binaries require Armv8.2 from 2016 (available in Raspberry Pi 2 >=2) and x86 binaries run on hardware from around 2010 (SSE4.2 + pclmul* instructions for fast CRC). We also build (but don't test using CI) binaries for Armv8.0 and SSE2-only systems. A quick install script downloads and unpacks the right binary for the target host. I find it generally hard to strike a good balance between backwards compatibility and usage of modern CPU features in newer AArch64 generations (https://en.wikipedia.org/wiki/AArch64 https://en.wikipedia.org/wiki/AArch64). We found that there are surprisingly many institutions on a shoestring budget (universities in emerging countries) or hobbyists that can't afford to upgrade their hardware. On a technical note, what I found quite cumbersome is that the cpu flags in /proc/cpuinfo don't always correspond with the flags passed as -march= to the compiler, e.g. "lrcpc" vs "rcpc". To make all of this work, one really needs to maintain two sets of flags.
- WirelessGigabit 3y ago
  I think in those cases it actually serves all your customers to do multiple builds so that they can pick and choose the one that matches their architecture the closest.
vkaku 3y ago
clang has been broken for a while in the last few versions. Many issues were left unfixed and development moved to 17.0.0 when they should have fixed those as point releases for 16.0.x instead (patches were available and not integrated). In this particular case though, the end processor/native detection seems to be failing and clang feature detection gets armv7l as native (or could just be the default generation option). Looks like a good bug to report, if only we get the good clang folks who will take the time to land a fix. I have been playing around with zig. My current focus will be on not using broken compiler backends for a while.
1vuio0pswjnm7 3y ago
"I guess nobody still runs these old things anywhere?" I have one running BSD UNIX-like OS as I type this comment.
sovietmudkipz 3y ago
Oh cool so this is kinda how one might debug why a program isn’t running on arm. I have a Unity Linux build that I can’t get running inside a container. Unity mono is trying to make a system call that isn’t available, even after passing in the amd64 flag to docker when running the container. I haven’t debugged it because I found a work around (enable development mode, change build settings so mono isn’t used). I should return to it at some point, just to learn more.

375 points by FPGAhacker 3y ago

frizlab 3y ago

Title is misleading

eimrine 3y ago
Because "as a default" statement is missing?
- jenadine 3y ago
  Yes. It now sounds like it is completely broken. But you can just fix it with a flag. And the change of default was probably an unintentional bug.
usr1106 3y ago
Aka called clickbait. Although the bug and the workaround are useful to know for everyone working with that machine.
blahgeek 3y ago
Yes. It should be “clang does not correctly detect host architecture in raspberry pi B+”

johnklos 3y ago

You'll see a whole bandwagon of people saying things like, "supporting old hardware is BAD! It takes time and money that nobody has!", as though someone needs to be hired to sit around and do nothing but pore over code and constantly rewrite code for old hardware. There's plenty of evidence to the contrary, but since when has evidence mattered when it comes to defending the right of big business / big distro to do whatever they want? ;) Really, this is just laziness and sloppiness on the Linux distro makers' part. Any amount of testing would catch this. Thanks, Rachel!

Elucalidavah 3y ago
> sit around and do nothing but pore over code and constantly rewrite code for old hardware In case of refactoring / restructuring, that's exactly so. But "drop support for this old hardware" is meant to be an intended decision with a clear deprecation warning, not accidentally.
- atemerev 3y ago
  The bazaar doesn’t work like this. There are no incentives to support old hardware. If there are enough people with old hardware, their activity might be enough to do something. Otherwise — puff, gone. Open source is already a miracle. The fact that something works somewhere is a miracle. I don’t tempt the powers and I don’t demand even more miracles to satisfy some perfectionist urges.
  - deleted 3y ago
    [deleted]
wmf 3y ago
I think RPi has struck a reasonable balance where the mainstream Linux community doesn't really support ancient ARMv6 so RPI themselves maintains forked software (e.g. Raspbian). This way the cost of legacy is borne by those who benefit from it, not everyone.
- deaddodo 3y ago
  Raspbian existed well before ARMv6 support dropped off. It's been their main distro from the outset, but mainstream distros with ARM builds only removed support for ARMv6 in the last 1-3 years (depending on distro).
  - account42 3y ago
    Pretty sure Debian's armhf images have always required ARMv7 or at least for much longer than 3 years. There are also armel images but those don't use hardware floats which makes them much less performant than what the original Pi is capable of. Pretty sure that that mismatch is why raspian exists in the first place.

teddyh 3y ago

It would probably be helpful to know the output of the command “dpkg-architecture” and the contents of the file “/etc/os-release”. Otherwise it will be hard to make any useful comments.

deleted 3y ago
[deleted]
cbmuser 3y ago
Exactly. The article omits information that is fundamental to being able to fix this problem.

ninepoints 3y ago

[flagged]

ajb 3y ago
It's not explicitly stated but in her reproduction she is actually running it on the pi, and compilers very much do normally compile for the arch that they are running on by default. A compiler which compiles for a different arch is called a cross-compiler. Unless you explicitly asked for a cross compile, it is indeed surprising for a compiler to emit a binary that won't run on the same machine.
- P-Nuts 3y ago
  It’s explicitly stated: “I figured, okay, maybe it's doing some optimization because it was built on the 4B. So, I went and did a build on the B+ natively. It also broke.” So she was originally compiling on a newer RPi, but immediately switched to compiling on the same one that couldn’t run it. And I believe the actual system images for RPi are built to support all models (there aren’t separate downloads for different models unless you want 64-bit) so it’s not crazy to think that everything would work building on a newer model. This is just a packaging defect, Clang/LLVM isn’t being configured correctly by RPiOS (Raspbian).
- thaumasiotes 3y ago
  > It's not explicitly stated but in her reproduction she is actually running it on the pi This is what the post says: >> This used to work in the old version (bullseye). It now breaks in the current one (bookworm). I figured, okay, maybe it's doing some optimization because it was built on the 4B. So, I went and did a build on the B+ natively. It also broke. If that's not an explicit statement that she's compiling on the target platform, what is?
wmf 3y ago
It sounds like clang running on the RPi 1 generates code that doesn't run. Usually compilers default to targeting whatever ISA it's running on but that doesn't seem to be the case here.

schemescape 3y ago

I didn't see it addressed here or in the article: this is a bug, right? Edit: oddly, after searching LLVM bugs, I found a bug that sounds pretty much exactly like this issue... but it's from 2012 and is closed (although the final couple of comments make it sound like maybe it wasn't actually fixed--note: I only skimmed the comments and I probably misunderstood): https://github.com/llvm/llvm-project/issues/13989 https://github.com/llvm/llvm-project/issues/13989 Edit again: I forgot about the comment at the end of the article that clarifies that explicitly passing the target results in a working program. In that case, it sounds like some sort of configuration bug--I would assume (but am not certain) that the default target would be the current processor, at least on Unix. That bug I linked was probably about producing incorrect code even when the target was set correctly which, thankfully, isn't happening today.

deleted 3y ago
[deleted]
wmf 3y ago
Yeah, this is not the behavior people expect.
- NikkiA 3y ago
  I would expect a arm64 machine to not build a arm32 compatible binary by default, it's the same as running clang on a x86-64 host and expecting it to produce 386 compatible binaries without a -march=i386 somewhere. The weird clang install on a fresh B+ install is more puzzling, unless there's some user error somewhere.
deleted 3y ago
[deleted]
Arnavion 3y ago
Yes, your bug is about the compiler emitting armv7 instructions despite being told to target armv6. Rachel fixed her problem by telling the compiler to target armv6. So I assume your bug is indeed already fixed and not related to Rachel's problem.
3y ago

deleted 3y ago

[deleted]

stephen_g 3y ago

Wow, looking at the history of the ARM generation the original versions of the Raspberry Pi uses, it’s hard to believe it’s so old! When the Raspberry Pi B+ was released (2014), the ARM core it used was already 11 years old (using the ARM1176 core from 2003). So it’s not unbelievable that you might need to start supplying an arch flag to produce compatible code building on a different platform (like the newer Raspberry Pi the article says they first built on). As others have said, it does seem like a misconfiguration (perhaps in the defaults shipped by their distribution) that the correct arch is not picked by default when building on the Raspberry Pi B+ itself.

yjftsjthsd-h 3y ago
> When the Raspberry Pi B+ was released (2014), the ARM core it used was already 11 years old (using the ARM1176 core from 2003). IIRC the original Pi used leftover chips from a TV box, which is the kind of product that IME never ships more compute than they have to, for price reasons.
- einr 3y ago
  TV boxes IME usually ship with less compute than they have to [in order to provide reasonable UX] ;)
  - yjftsjthsd-h 3y ago
    Eh, the result may suck, but I don't think it's usually a hardware problem.
  - adhesive_wombat 3y ago
    Anything to satisfy whichever law it is that says "lagginess remains constant".
    - actionfromafar 3y ago
      "Huh, this CPU is kind of snappy running native code now. What should we do?" "Let's move application development to Python then, I suppose." "Thanks, that fixed it." Probably what happened in my 55" smart TV dev team.

dang_rachel 3y ago

[dead]

dang_rachel2 3y ago

[dead]

matja 3y ago

clang/clang++ read from /etc/env.d/gcc to get the target flags/profile, it's up to the OS to maintain them and make sure they're correct, looks like that didn't happen for this OS. My Gentoo ARM SBC based on an even more ancient armv4 arch has been chugging along just fine with the latest gcc/clang updates: grep CTARGET /etc/env.d/gcc -r /etc/env.d/gcc/armv4tl-softfloat-linux-gnueabi-11.3.0:CTARGET="armv4tl-softfloat-linux-gnueabi"

contingencies 3y ago
Gentoo always works .. it just takes longer :)
Arnavion 3y ago
/etc/env.d is a Gentoo-specific directory to define default env vars for user sessions. It's not a feature of clang to read that directory, so it's not correct to assume other distros would have it. It's just that Gentoo's compiler setup reads the CTARGET env var to select the target, and Gentoo uses /etc/env.d to set it.
- matja 3y ago
  Is that why other distros break? :)

auselen 3y ago

Confused article? You make a host/native build instead of cross and expect it to work on some other machine?

daviddoran 3y ago
No. Half way through the article she specifically starts doing everything on the B+ (the old RPI with the issue).
- auselen 3y ago
  Thanks for that. I didn’t notice she switch to B+ later.

deleted 3y ago

[deleted]

opello 3y ago

Seems like the problem is likely a configuration target change in the clang-13 package that's current for bookworm. Specifically because under bullseye (and clang-11) the default target is armv6k-unknown-linux-gnueabihf while under bookworm (and clang-13) the default target is arm-unknown-linux-gnueabihf. Or maybe the default changed for the given build configuration on the LLVM side?

opello 3y ago
I really wish I understood the Debian change management process better. I guess I don't even really know if Raspbian is actually maintained by Debian. But, when comparing [1] to [2], the rules file has a nice test that says "if DEB_HOST_ARCH is armhf, set the LLVM_HOST_TRIPLE to armv6k..." which seems to confirm a build configuration change. [1] http://raspbian.raspberrypi.org/raspbian/pool/main/l/llvm-toolchain-11/llvm-toolchain-11_11.0.1-2~deb10u1+rpi1.debian.tar.xz http://raspbian.raspberrypi.org/raspbian/pool/main/l/llvm-to... [2] http://raspbian.raspberrypi.org/raspbian/pool/main/l/llvm-toolchain-13/llvm-toolchain-13_13.0.1-6~deb10u4+rpi1.debian.tar.xz http://raspbian.raspberrypi.org/raspbian/pool/main/l/llvm-to...
- orra 3y ago
  To answer your incidental question, Raspian is maintained by Raspberry Pi folks, not Debian.
  - opello 3y ago
    Ah, thanks! I dug a little more and found that the original test [1] from llvm-toolchain was a little different in the upstream Debian repository. It set the triple to armv7l for armhf hosts. But I haven't yet found a similar repository on the Raspbian side. I guess I'd expect to find it within their GitHub org, but my searching didn't reveal it. [1] https://salsa.debian.org/pkg-llvm-team/llvm-toolchain/-/commit/af56187b7d04d718f3f2229691885b102bb026a5 https://salsa.debian.org/pkg-llvm-team/llvm-toolchain/-/comm...
    - 3y ago

JonChesterfield 3y ago

I doubt this is a deliberate change. Picking up information from the environment - a sibling mentions /etc/env.d/gcc - seems fairly likely. I'd guess the default triple is something like arm-unknown-linux unless clang finds or is is told something more specific to use, and the mechanism by which it gets told to use something more specific has fallen over. This might mean there are no arm v6 buildbots running, or it might mean there are ones running but the implicit configuration is still working on them. LLVM is a really good cross compiler. Build for any target from any target, no trouble. Clang is less compelling - if it's built with the target, and you manage to tell it what target to build for, it'll probably do the right thing (as in this post - it guessed wrong, but given more information, did the right thing). Then the runtime library story is worse again - you've built for armv4 or whatever, but now you need to find a libc etc for it, and you might need to tell the compiler where those libraries and headers are, and for that part I'm still unclear on the details.

rcarmo 3y ago
Most distros and compilers effectively dropped ARMv6 a couple of years back - I had similar trouble building binaries for my old Synology NAS.
- anthk 3y ago
  Alpine Linux might still support it I think.
hulitu 3y ago
> Picking up information from the environment - a sibling mentions /etc/env.d/gcc - seems fairly likely. Why would CLANG do this ?
- elteto 3y ago
  Clang already replicates a bunch of flags, macros, and behaviors from gcc. The objective is to be a drop-in replacement, and make the developer experience much nicer when migrating. There are some rough corners, of course, but overall it’s actually very nice.

fsniper 3y ago

Title is unfortunately sensational. This is a default target change. Turns out clang still can build binaries for the Pi B+. You just need to be explicit about the architecture. So perhaps a small title change that's more clear about this being only default setting change?

nottorp 3y ago
Doesn’t seem so sensational when it can’t build binaries for the target machine… on the target machine itself…
- cbmuser 3y ago
  It’s sensational because it’s wrong. LLVM still supports even ARMv5T which is the baseline of Debian’s armel port.
- dagmx 3y ago
  It can, it just doesn’t by default. Which is what the person you’re replying to is saying.
  - nottorp 3y ago
    And... does it make sense to you... when you're not cross compiling?
    - dagmx 3y ago
      The point is that it objectively CAN compile to the right target. The capability is not broken. It however DOESNT due to a configuration bug. Therefore it doesn’t have to make sense because it’s clearly not intentional. your sentence saying “it can’t build” is therefore incorrect. It’s the distinction between the two capitalized words above.
      - brian-armstrong 3y ago
        If "clang helloworld.c" doesn't produce a working a.out out of the box, I think it's fair to say builds are broken. Plenty of projects won't build in those circumstances without some assistance.

draw_down 3y ago

[dead]

cbmuser 3y ago

The article doesn’t mention whether Debian or Raspian was installed. And, in case of Debian, whether the armel or armhf port is being used. Without that information, it’s pretty pointless to make claims about the instruction set LLVM compiles to because that’s a matter of what native target LLVM has been configured for. FWIW, in Debian, llvm-toolchaim-snapshot still supports armel which uses ARMv5T as the baseline (there is currently an unrelated bug in LLVM’s OpenMP library though which prevents a successful build).

jchw 3y ago
What's weird is that the Clang binary is clearly compiled for an instruction set that is compatible with the Pi B+, but it doesn't target an instruction set that is compatible with the Pi B+. This is genuinely weird, since that's not meant to be a cross-compiler; in theory, the host and the target should be the same. Presumably the image is Raspbian. I don't see a reason why not to assume that.
- guipsp 3y ago
  One key thing is missing from your comment (which explains the 'weirdness') - clang, and other llvm-family tools are cross-compatible by default. There is no separate cross compilation binary. This is just a configuration bug.
  - jchw 3y ago
    Yes, that's true, although it doesn't actually explain the weirdness. I compile Clang all the time and there's no obvious reason why you'd get cross-compiled binaries out of Clang if you just compile and install it normally. The bug, configuration or otherwise, is the weirdness.

rschu1ze 3y ago

The database I work on (ClickHouse) tries hard to stay compatible with really old hardware. The standard ARM binaries require Armv8.2 from 2016 (available in Raspberry Pi 2 >=2) and x86 binaries run on hardware from around 2010 (SSE4.2 + pclmul* instructions for fast CRC). We also build (but don't test using CI) binaries for Armv8.0 and SSE2-only systems. A quick install script downloads and unpacks the right binary for the target host. I find it generally hard to strike a good balance between backwards compatibility and usage of modern CPU features in newer AArch64 generations (https://en.wikipedia.org/wiki/AArch64 https://en.wikipedia.org/wiki/AArch64). We found that there are surprisingly many institutions on a shoestring budget (universities in emerging countries) or hobbyists that can't afford to upgrade their hardware. On a technical note, what I found quite cumbersome is that the cpu flags in /proc/cpuinfo don't always correspond with the flags passed as -march= to the compiler, e.g. "lrcpc" vs "rcpc". To make all of this work, one really needs to maintain two sets of flags.

WirelessGigabit 3y ago
I think in those cases it actually serves all your customers to do multiple builds so that they can pick and choose the one that matches their architecture the closest.

vkaku 3y ago

clang has been broken for a while in the last few versions. Many issues were left unfixed and development moved to 17.0.0 when they should have fixed those as point releases for 16.0.x instead (patches were available and not integrated). In this particular case though, the end processor/native detection seems to be failing and clang feature detection gets armv7l as native (or could just be the default generation option). Looks like a good bug to report, if only we get the good clang folks who will take the time to land a fix. I have been playing around with zig. My current focus will be on not using broken compiler backends for a while.

1vuio0pswjnm7 3y ago

"I guess nobody still runs these old things anywhere?" I have one running BSD UNIX-like OS as I type this comment.

sovietmudkipz 3y ago

Oh cool so this is kinda how one might debug why a program isn’t running on arm. I have a Unity Linux build that I can’t get running inside a container. Unity mono is trying to make a system call that isn’t available, even after passing in the amd64 flag to docker when running the container. I haven’t debugged it because I found a work around (enable development mode, change build settings so mono isn’t used). I should return to it at some point, just to learn more.