10 ms·

Weird architectures weren’t supported to begin with (2021)

107 points by dbaupp 4y ago

DerekL 4y ago
Title needs (2021).
CJefferson 4y ago
I've been trying to officially mark some open source software I work on as not supporting big endian. We don't test it, and I think it's very likely there are some subtle bugs. Sometimes someone will ask for support, but (unsurprisingly, I don't blame them) they don't want to put the work into testing up to the standard we achieve on ARM and x64.
- cbmuser 4y ago
  IBM zSeries runs Linux in big-endian mode and is still a mainstream product. If your code doesn’t work on big-endian, it’s usually poorly designed.
  - LegionMammal978 4y ago
    Eh, I think it's a stretch to say that any possible dependency on little-endian can be rounded down to "poor design". If you're operating with a format or protocol that's unconditionally little-endian, then it takes strictly less work to operate on the data structures in place, or on direct bitwise copies of the data structures, rather than conditionally transforming the values to native-endian, operating on them, and conditionally transforming them back to little-endian. You can argue that everyone's code ought to take the longer route due to the existence of big-endian systems, but that's what this blog post is arguing against.
  - pm215 4y ago
    Depending on what your code is doing, even if there are good endian-agnostic abstractions, endianness bugs are still likely to slip in from time to time unless you're defending against them by having your CI run on a big-endian platform. QEMU does this, and although CI-fails-on-BE is rare, it happens often enough that I wouldn't want to lose the coverage. (On the theme of the original article, IBM actively support this by giving us CI resource via the IBM LinuxONE Community Cloud machines, as well as being active in upstream development.)
  - cesarb 4y ago
    > > I've been trying to officially mark some open source software I work on as not supporting big endian. We don't test it, and I think it's very likely there are some subtle bugs. > IBM zSeries runs Linux in big-endian mode and is still a mainstream product. For most hobbyist open source software developers, it's a niche product in practice. I can easily test on 32-bit and 64-bit x86 (most desktops and laptops can run it natively); I can easily test on 32-bit and 64-bit ARM (a Raspberry Pi 3B or newer is a common enough device, and using Termux on a phone is also an option); but how would one get access to an IBM zSeries? > If your code doesn’t work on big-endian, it’s usually poorly designed. The parent comment mentioned "subtle bugs". It's not hard to accidentally introduce byte order dependencies; one simple example (though from the big endian side) would be forgetting to use "htons" on a port number. Since more and more protocols and file formats are natively little endian nowadays, it's easy to miss a conversion between "little endian" and "native endian", since it will still work perfectly on little endian architectures.
    - matja 4y ago
      > but how would one get access to an IBM zSeries? qemu-system-s390x - I run Linux in it for some automatic tests, finds some funny bugs occasionally.
      - rbanffy 4y ago
        Also an option on Travis CI. Another option is getting Hercules, which can emulate a 390x, but, IIRC, falls short on some post z14 features more recent Linuxes rely upon. If you need time on an actual machine, there is the LinuxONE community cloud (Linux under z/VM) and IBM provides hardened (and expensive for 24x7 operation) Linux on Z instances (under KVM). The IBM mainframe developer relations team is unusually friendly and approachable as well.
  - CJefferson 4y ago
    Any code which cares about splitting ints, pointers or floats into individual bytes, or doing network traffic, will have to care about endianness. Now, there are of course ways to handle this, but in C and C++ it's easy to make mistakes, and it's hard to test. I'd be happy supporting big-endian if someone wants to buy me a z series equivalent in power to a mid-level AMD laptop, and also get GitHub to add them as a standard target for GitHub actions.
    - johnklos 4y ago
      Or you could set up a Raspberry Pi with NetBSD/aarch64eb and just let it take its time.
  - IshKebab 4y ago
    Poorly designed... or you don't want to have to think about Endianness just to support some weird niche architecture that only a handful of enormous corporations use? I really don't think there's anything wrong with assuming little endian in this age. It's like assuming 8 bit bytes, or 2s complement signed numbers.
wizzard0 4y ago
One more example that security cannot be abstracted, packaged and bought in a box. The same goes for performance (but that one can be packaged at least sometimes)
cwzwarich 4y ago
> The C abstract machine, despite looking a lot like a PDP-11, leaks the underlying memory and ordering semantics of the architecture being targeted. The result is that even seasoned C programmers regularly rely on architecture-specific assumptions when writing ostensibly cross-platform code: assumptions about the atomicity of reads and writes, operation ordering, coherence and visibility in self-modifying code, the safety and performance of unaligned accesses, and so forth. Each of these, apart from being a potential source of unsafety, are impossible to detect statically in the general case: they are, after all, perfectly correct (and frequently intended!) on the programmer’s host architecture. I don't disagree with his general point that C isn't really cross-platform, but these are bad examples: - The C memory model addresses the issues regarding atomicity, memory ordering, and the safety of unaligned accesses (although users might not like the answer for the last one). - Coherence within a C program is guaranteed by the language, so issues with coherence only arise when interacting with an incoherent external agent. How could any language solve this? - Are there any examples of self-modifying code that don't already make assumptions about the underlying ISA? If you're already doing that, dealing with data/instruction memory consistency doesn't seem particularly onerous. Even when only targeting AArch64, it isn't possible to write platform-independent userspace assembly code that does this because instruction cache maintenance instructions might not be enabled for EL0.
- deleted 4y ago
  [deleted]
ris 4y ago
I never agreed with this. Who defines what is considered "weird"? Should all projects declare up-front that it only is designed to work on: - x86_64 and aarch64 - linux, darwin and windows - glibc (no musl) - binutils x.xx - ... ... where do you stop before you're defining an exact linux distribution? If it weren't for people building and using packages for architectures that are "unsupported", I should point out that there would be no good userspace support for ARM or aarch64. These too were once "weird architectures" that only have good support now because of people building for (and using) these architectures without permission, then shaking all the bugs out. This is currently still happening for risc-v. I have to contrast this attitude to CPython, who, while I'm sure they never originally said that their code was designed to run on s390 (risc-v, pick your architecture..), would not go ahead with a change that froze out these users. I also don't think the "we're just a poor bunch of small-time package maintainers whose package accidentally became popular" angle quite works, because they call themselves the "Python Cryptographic Authority" and use the prominent package name `cryptography`, which certainly appears to imply some sort of official status. I wouldn't blame people for inferring that its support policies were somewhat aligned with the python foundation's.
- jackmott 4y ago
  [dead]
- 323 4y ago
  > I have to contrast this attitude to CPython, who, while I'm sure they never originally said that their code was designed to run on s390 (risc-v, pick your architecture..), would not go ahead with a change that froze out these users. You have a wrong recollection of history. CPython removed support for several platforms, including former major ones like Windows XP and Vista. > Attached PR removes code to support Windows Vista. https://bugs.python.org/issue32592 https://bugs.python.org/issue32592 And Python 3.9+ doesn't work anymore on Windows 7: https://discuss.python.org/t/windows-7-support-for-python-3-9-or-3-10-without-a-fork/13842 https://discuss.python.org/t/windows-7-support-for-python-3-...
  - tyingq 4y ago
dwheeler 4y ago
A lot of the problem is that GCC hasn't supported Rust, and LLVM doesn't support many platforms. Since this article was written (2021), there's been work to implement Rust fully in GCC, as well as ways to call from LLVM the GCC backend (which supports far more architectures). So there's a reasonable expectation that some of this problem will be fixed in the long-term. IBM mainframes write most paychecks, and companies do pay some OSS developers to support some of the architectures the author doesn't like. Making it impossible is different.
johnklos 4y ago
That's an interesting take. It seems like a modern slant on the old trope of "all the world's a VAX", then "who cares if it's not i386?", and now, "of course it's portable - we support both kinds - amd64 AND aarch64". But it's a bit disingenuous. Are developers being overwhelmed by problem reports for things that hardly matter? Looking at NetBSD's pkgsrc, which supports more OSes than just NetBSD, we see about 64,000 patch files for about 26,000 packages. While many packages don't require any patches, some require many. This is because for many programs, upstream just don't care. It's not just about odd architectures - they just don't care about NetBSD support, or support for different, alternative OSes. Perhaps that's fine, but consider this - the patches already exist and are tested, and they're maintained in pkgsrc. There are times people become indignant about being told how to fix their software for use under systems other than Windows, common Linux and macOS, on CPUs other than amd64 and aarch64. Yes, indignant. Some Python folks don't want to consider anyone would want to use Python on systems (like embedded) that don't have full IEEE floating point emulation. postgresql would rather mark platforms as broken than just let them compile the standard c portions of their code. People like to say, "they don't want to spend the time and energy", as if it adds work. No - there are examples where people choose the broken path when the not broken path is just as easy if not easier. Nobody is obligated to add or patch anything, even if the patches are freely given and well tested. But the fact that some people actively fight against portability is disturbing, and should be viewed with suspicion. The author of the article seems to have forgotten some history. They mention downsides of the c ecosystem, like the lack of standardized ways to build, the lack of consistent package management, and so on. But in every instance where those things have been imposed, the imposition has been problematic, has it not? Have we not seen security issues with PyPi? Have we not seen the dependency hell which is Ruby? The point is that nobody can tell anybody else how to do open source, so we'll always have a hundred different ways to do things. We also can't help that some people will be gatekeepers. But we personally can ignore the gatekeepers and help make things portable. Making excuses for why portability is somehow extra work is only encouraging gatekeeping. We don't need to do that - gatekeepers already have enough energy on their own.
pornel 4y ago
Are there architectures which LLVM doesn't support, which aren't retrocomputing or 16-bit microcontrollers?
- Denvercoder9 4y ago
  One example is the Xtensa architecture, used in the 32-bit ESP8266/ESP32 microcontrollers that are fairly popular with hobbyists (though Espressif seems to be moving to RISC-V for their newer offerings).
  - pornel 4y ago
    Yeah, the RISC-V version seems to be supported: https://lib.rs/crates/esp8266 https://lib.rs/crates/esp8266
    - Denvercoder9 4y ago
      That crate requires a fork of the Rust compiler to use.
- monocasa 4y ago
  Mainline LLVM still doesn't support Xtensa or ARC AFAIK. ESP32 was Xtensa until their recent RISC-V based designs.
PaulHoule 4y ago
Personally C for Arduino programming drives me up the wall. With 32 general purpose registers in assembly language you can often keep all the variables in your inner loop in registers and still have a few left over for the interrupt handler. In C on the other hand you know it is moving the stack pointer around meaninglessly just so it can support recursive functions which aren’t really appropriate for embedded systems. I still write C for it because as beautiful as AVR-8 is, it is a dead end and if I need more capability I can take my C program to an ARM and maybe someday a RISC-V board. (I dream of embedding a soft AVR-8 on an FPGA with some special logic but the engineering students I know tell me that the Verilog class was like getting mauled by a bear.). At least gcc meets me halfway and has 24 bit ints in AVR-8. For that matter I just got a VisionFive 2 board that I need to put in a case and bring up with Linux. It has been a long time since I had to mess with other peoples C programs and bring them up on a new architecture but I think I’ll be doing it again.
phkahler 4y ago
Yeah why should we even care about s390 for some things? https://github.com/solvespace/solvespace/issues/1264 https://github.com/solvespace/solvespace/issues/1264 I don't think big commercial customers are designing airplanes with it.
- cesarb 4y ago
  That github issue is about the 64-bit s390x. The "weird architecture" being talked about here is the older 31-bit s390.
Karellen 4y ago
> Imagine, for a moment, that you’re a maintainer of a popular project. [...] You’ve also got a CI/CD pipeline that produces canonical releases of your project on tested architectures; [...] Because your project is popular, others also distribute it: Linux distributions, third-party package managers, and corporations seeking to deploy their own controlled builds. > You don’t know about any of the above until the bug reports start rolling in: users will report bugs that have already been fixed, bugs that you explicitly document as caused by unsupported configurations, bugs that don’t make any sense whatsoever. If 3rd-party build recipients are sending their bugs directly to you, that's a failure of the 3rd-party builders to take responsibility for their packages. They should be telling you to submit your bugs to them, so they can check their packaging, and then the packagers should talk to upstream only if there are issues to be resolved there. > You struggle to debug your users’ reports, since you don’t have access to the niche hardware, environments, or corporate systems that they’re running on. Yes, C supports lots of environments, and Rust supports quite a few (and hopefully more, soon), but I think it's perfectly fine for an upstream author to only support a subset of those. The benefits of Free Software is that if you want to get some software running on platforms the original author doesn't support but the language does, you can do that. Investigate the bug yourself. Figure out if it's in the app, or a 3rd-party library, or even the toolchain. Submit a patch. A good proportion of authors will happily accept patches for systems they themselves can't test on, if it's not too intrusive, and doesn't cause regressions on the platforms the author does support. They might be willing to entertain an intrusive patch series that allows for better cross-arch support, if you're willing to work with them on that. And if an author is not interested in helping you scratch your itch (ew! :-), create a fork. That should be easier than ever these days with the version control tools we have now, so much more than when Free Software was first envisioned. The author may even be willing to point other people who want to use their software on your platform your way (e.g. in their README), if only to get those users of their back!
- slondr 4y ago
  > The benefits of Free Software is that if you want to get some software running on platforms the original author doesn't support but the language does, you can do that. The exact cause of this entire controversy is that the software in question is switching to a language that supports less platforms.
ergonaught 4y ago
> As someone who likes C: this is all C’s fault. Really. I think I agree with almost everything in this post except this sentiment. Blaming the tool is silly. You may as well blame assembly language, and if you're doing that you may as well blame CPUs and electricity and you know what, physics is harmful, we should rewrite it in Rust.
- celeritascelery 4y ago
  If C had a standard way to test, build, and distribute packages then may of the authors concerns would be resolved. But it doesn’t, and other languages do.
jameshart 4y ago
There’s always been a weird double standard in open source software where it has been seen as ‘reasonable’ to expect the same code to compile and work on everything from an Arduino to an IBM mainframe, PDP11 to a RISC-V. ..But getting it to work on Windows? What? Do you expect the developers to fork out for Windows licenses? Don’t be ridiculous. Heck, some software is still downright snooty about the absurd idea that it should run cleanly on a Mac.
- cesarb 4y ago
  Playing devil's advocate: since the operating system mostly abstracts the underlying hardware, when programming with a high-level language, an IBM mainframe running Linux and a MIPS router running Linux are both closer to an AMD64 desktop running Linux than the same AMD64 desktop running Windows. And Windows is a particularly annoying case, since its native API is so different from the Linux native API (and details leak even when using API wrappers, for instance removing a recently-created file can fail on Windows because some other process like an antivirus grabbed a handle to it). But yes, I agree that a nontrivial fraction of the objections to making software run on Windows (or Mac) is for ideological reasons.
- otabdeveloper4 4y ago
  Microsoft has too many billions of dollars to count. Maybe they should be the ones to be held accountable for upholding community standards, not the free volunteers.
  - jameshart 4y ago
    Which ‘community standards’ do you mean? POSIX?
    - otabdeveloper4 4y ago
      POSIX is a corporate standard. Community standards are usually not formalized.
- pohl 4y ago
docandrew 4y ago
I think the frustrating thing is that, while rust may consider one of these architectures “tier 2” or unsupported or whatever, there’s a Python interpreter that presumably doesn’t, and whoever is using the Python cryptography package expects it to work the same place all their other code does. And very likely it’s some dependency several layers deep. If my code is going to end up architecture-dependent then why bother with an interpreted language in the first place?
- StreamBright 4y ago
  > why bother with an interpreted language in the first place +1 As a person who uses Python for 12 years professionally I usually try to avoid it as much as I can. When you have a Python problem you usually have a C, C++, Rust, libc, arch problem that you do not realise. Most of the useful parts of Python are written in C, C++, Fortran, Rust so when you try to deploy it some less frequently used platform it can burst into flames the worst kind of ways. You can try to deploy the AWS Lambda / Python 3.9 and see the lolz. I have spent more hours on trying to get some Python lib work on a platform than learning Rust. I think interpreted languages are a dead end especially with bad practices. I make my living writing Python but it is a misery every direction. It is not an accident that Rust is the most loved language continuously because it just works. Anything I try to do in it works as expected and I am not walking on a mine field. Let me give you a simple example how Python can surprise you. Lets create an app in Python that uses a lib called X. You build your project and everything works locally, unit tests are ok, integration tests too and so on. Now deploy this code to AWS Lambda (not your choice, employer decided to go with that). You package everything up on a Linux that matches the architecture of the target (lets say X64). If you are not familiar with setting the target Python version with pip (and most documentation does not mention that for Lambda) you deploy your code and try to invoke it. Fail. You have X.311.so in your deployment package and Lambda tries to load X.39.so. Now you need to figure out how to set the version or have a build env that matches the CPU AND the Python version with the target system. I could continue this rabbit hole for some more but the point is that you can't talk about Python alone, you need to pull in the Cartesian product of libc, libXX, C, C++, Fortrant, all the compilers for these, cpu architectures and Python versions. On a lucky they you might have a working system. With Rust everything just worked the first time we tried to use it. I could not believe it how easy it was to put out a working system at the first try. It only beats Python by an order of magnitude in terms of performance but the amount of effort it took us to deploy it was also much less.
pyuser583 4y ago
If only we had stuck with Ada.
rbanffy 4y ago
> Give up on weird ISAs and platforms NEVER!
- Hackbraten 4y ago
  I didn’t interpret it as “say goodbye to niche arch support entirely” but rather as “it’s about time we stop externalizing to the OSS volunteer community the huge burden of supporting weird platforms, let’s rather shift that labor and cost onto the actual stakeholders of those architectures.” That doesn’t imply end users of niche architectures are supposed to lose their favorite apps.
  - cbmuser 4y ago
    If you look at the market shares, you’ll quickly realize that this would mean abandoning all the BSDs, Haiku, Hurd, all the obscure Linux distributions and so on. Just support macOS, RHEL/SLES, Debian, Ubuntu and Windows. Everything else gets tagged with “UNSUPPORTED/WONTFIX”. But please don’t complain when your favorite operating system is not supported. Or, you know, we could just stop trying to tell others what targets to use. And, FWIW, Rust is actually getting support for more architectures thanks to the GCC codegen and GCC frontend.
    - dwheeler 4y ago
      Why so many architectures? Ensure your software can only run on Windows: https://gs.statcounter.com/os-market-share/desktop/worldwide https://gs.statcounter.com/os-market-share/desktop/worldwide :-). Vive la difference.
    - sophacles 4y ago
      You're welcome to do it that way for your software. Others are allowed to choose some subset or superset of your choice. If they don't like your choice, they are allowed to fork the software to support what they want. Case in point - the gcc projects for rust are not officially supported by rust. They are alternative rust compilers - the authors of those have explicitly stated they will follow the features (etc) of the official rustc, and maybe the rust language team will consider those other compilers when designing new features, but it's not on the rust language team to do so. Another case in point - many projects don't support most linux distros. The job of the linux distro is maintain a fork (usually in the form of patches against upstream in the source package) of the software that works well with their system.
zamalek 4y ago
> The security of a program is a function of its own design and testing, as well as the design, testing, and basic correctness of its underlying platform: everything from the userspace, to the kernel, to the compilers themselves. You don't even have to worry about bugs. There are many cryptography primitives that require constant-time to guarantee security. That means that the latency of each machine code instruction could be vital to security. The latency of instructions between vendors within the same architecture is already a concern.
- woodruffw 4y ago
  This is a good point! But to be maximally precise: variations between each ISA implementation's latencies is not particularly important; what's important is that instructions on data-dependent paths are not themselves data-dependent in timing. In other words: it's okay if AMD64 takes 9 cycles to execute an instruction while IA32e takes 7; the problem arises when AMD64 takes 9 + k cycles for k bytes of input while cryptographic engineers have tested on IA32e and assumed a constant overhead of 7 cycles.
  - Gordonjcp 4y ago
    > the problem arises when AMD64 takes 9 + k cycles for k bytes of input while cryptographic engineers have tested on IA32e and assumed a constant overhead of 7 cycles. Why is that a problem? Doesn't that just leak something you could determine by just looking at the CPU type?
    - bluGill 4y ago
      That also leaks how many bytes of data are being processed. This sometimes matters. When checking for a password match you have to check all characters in the.string otherwise you leak where the mismatch was. Even in a properly salted and hashed scheme that makes breaking the password easier.
    - jeroenhd 4y ago
      These side channel attacks allow perfectly secure algorithms to leak plaintext or even complete keys.

107 points by dbaupp 4y ago

Title needs (2021).

I've been trying to officially mark some open source software I work on as not supporting big endian. We don't test it, and I think it's very likely there are some subtle bugs. Sometimes someone will ask for support, but (unsurprisingly, I don't blame them) they don't want to put the work into testing up to the standard we achieve on ARM and x64.

cbmuser 4y ago
IBM zSeries runs Linux in big-endian mode and is still a mainstream product. If your code doesn’t work on big-endian, it’s usually poorly designed.
- LegionMammal978 4y ago
  Eh, I think it's a stretch to say that any possible dependency on little-endian can be rounded down to "poor design". If you're operating with a format or protocol that's unconditionally little-endian, then it takes strictly less work to operate on the data structures in place, or on direct bitwise copies of the data structures, rather than conditionally transforming the values to native-endian, operating on them, and conditionally transforming them back to little-endian. You can argue that everyone's code ought to take the longer route due to the existence of big-endian systems, but that's what this blog post is arguing against.
- pm215 4y ago
  Depending on what your code is doing, even if there are good endian-agnostic abstractions, endianness bugs are still likely to slip in from time to time unless you're defending against them by having your CI run on a big-endian platform. QEMU does this, and although CI-fails-on-BE is rare, it happens often enough that I wouldn't want to lose the coverage. (On the theme of the original article, IBM actively support this by giving us CI resource via the IBM LinuxONE Community Cloud machines, as well as being active in upstream development.)
- cesarb 4y ago
  > > I've been trying to officially mark some open source software I work on as not supporting big endian. We don't test it, and I think it's very likely there are some subtle bugs. > IBM zSeries runs Linux in big-endian mode and is still a mainstream product. For most hobbyist open source software developers, it's a niche product in practice. I can easily test on 32-bit and 64-bit x86 (most desktops and laptops can run it natively); I can easily test on 32-bit and 64-bit ARM (a Raspberry Pi 3B or newer is a common enough device, and using Termux on a phone is also an option); but how would one get access to an IBM zSeries? > If your code doesn’t work on big-endian, it’s usually poorly designed. The parent comment mentioned "subtle bugs". It's not hard to accidentally introduce byte order dependencies; one simple example (though from the big endian side) would be forgetting to use "htons" on a port number. Since more and more protocols and file formats are natively little endian nowadays, it's easy to miss a conversion between "little endian" and "native endian", since it will still work perfectly on little endian architectures.
  - matja 4y ago
    > but how would one get access to an IBM zSeries? qemu-system-s390x - I run Linux in it for some automatic tests, finds some funny bugs occasionally.
    - rbanffy 4y ago
      Also an option on Travis CI. Another option is getting Hercules, which can emulate a 390x, but, IIRC, falls short on some post z14 features more recent Linuxes rely upon. If you need time on an actual machine, there is the LinuxONE community cloud (Linux under z/VM) and IBM provides hardened (and expensive for 24x7 operation) Linux on Z instances (under KVM). The IBM mainframe developer relations team is unusually friendly and approachable as well.
- CJefferson 4y ago
  Any code which cares about splitting ints, pointers or floats into individual bytes, or doing network traffic, will have to care about endianness. Now, there are of course ways to handle this, but in C and C++ it's easy to make mistakes, and it's hard to test. I'd be happy supporting big-endian if someone wants to buy me a z series equivalent in power to a mid-level AMD laptop, and also get GitHub to add them as a standard target for GitHub actions.
  - johnklos 4y ago
    Or you could set up a Raspberry Pi with NetBSD/aarch64eb and just let it take its time.
- IshKebab 4y ago
  Poorly designed... or you don't want to have to think about Endianness just to support some weird niche architecture that only a handful of enormous corporations use? I really don't think there's anything wrong with assuming little endian in this age. It's like assuming 8 bit bytes, or 2s complement signed numbers.

wizzard0 4y ago

One more example that security cannot be abstracted, packaged and bought in a box. The same goes for performance (but that one can be packaged at least sometimes)

cwzwarich 4y ago

> The C abstract machine, despite looking a lot like a PDP-11, leaks the underlying memory and ordering semantics of the architecture being targeted. The result is that even seasoned C programmers regularly rely on architecture-specific assumptions when writing ostensibly cross-platform code: assumptions about the atomicity of reads and writes, operation ordering, coherence and visibility in self-modifying code, the safety and performance of unaligned accesses, and so forth. Each of these, apart from being a potential source of unsafety, are impossible to detect statically in the general case: they are, after all, perfectly correct (and frequently intended!) on the programmer’s host architecture. I don't disagree with his general point that C isn't really cross-platform, but these are bad examples: - The C memory model addresses the issues regarding atomicity, memory ordering, and the safety of unaligned accesses (although users might not like the answer for the last one). - Coherence within a C program is guaranteed by the language, so issues with coherence only arise when interacting with an incoherent external agent. How could any language solve this? - Are there any examples of self-modifying code that don't already make assumptions about the underlying ISA? If you're already doing that, dealing with data/instruction memory consistency doesn't seem particularly onerous. Even when only targeting AArch64, it isn't possible to write platform-independent userspace assembly code that does this because instruction cache maintenance instructions might not be enabled for EL0.

deleted 4y ago
[deleted]

ris 4y ago

I never agreed with this. Who defines what is considered "weird"? Should all projects declare up-front that it only is designed to work on: - x86_64 and aarch64 - linux, darwin and windows - glibc (no musl) - binutils x.xx - ... ... where do you stop before you're defining an exact linux distribution? If it weren't for people building and using packages for architectures that are "unsupported", I should point out that there would be no good userspace support for ARM or aarch64. These too were once "weird architectures" that only have good support now because of people building for (and using) these architectures without permission, then shaking all the bugs out. This is currently still happening for risc-v. I have to contrast this attitude to CPython, who, while I'm sure they never originally said that their code was designed to run on s390 (risc-v, pick your architecture..), would not go ahead with a change that froze out these users. I also don't think the "we're just a poor bunch of small-time package maintainers whose package accidentally became popular" angle quite works, because they call themselves the "Python Cryptographic Authority" and use the prominent package name `cryptography`, which certainly appears to imply some sort of official status. I wouldn't blame people for inferring that its support policies were somewhat aligned with the python foundation's.

jackmott 4y ago
[dead]
323 4y ago
> I have to contrast this attitude to CPython, who, while I'm sure they never originally said that their code was designed to run on s390 (risc-v, pick your architecture..), would not go ahead with a change that froze out these users. You have a wrong recollection of history. CPython removed support for several platforms, including former major ones like Windows XP and Vista. > Attached PR removes code to support Windows Vista. https://bugs.python.org/issue32592 https://bugs.python.org/issue32592 And Python 3.9+ doesn't work anymore on Windows 7: https://discuss.python.org/t/windows-7-support-for-python-3-9-or-3-10-without-a-fork/13842 https://discuss.python.org/t/windows-7-support-for-python-3-...
- tyingq 4y ago

dwheeler 4y ago

A lot of the problem is that GCC hasn't supported Rust, and LLVM doesn't support many platforms. Since this article was written (2021), there's been work to implement Rust fully in GCC, as well as ways to call from LLVM the GCC backend (which supports far more architectures). So there's a reasonable expectation that some of this problem will be fixed in the long-term. IBM mainframes write most paychecks, and companies do pay some OSS developers to support some of the architectures the author doesn't like. Making it impossible is different.

johnklos 4y ago

That's an interesting take. It seems like a modern slant on the old trope of "all the world's a VAX", then "who cares if it's not i386?", and now, "of course it's portable - we support both kinds - amd64 AND aarch64". But it's a bit disingenuous. Are developers being overwhelmed by problem reports for things that hardly matter? Looking at NetBSD's pkgsrc, which supports more OSes than just NetBSD, we see about 64,000 patch files for about 26,000 packages. While many packages don't require any patches, some require many. This is because for many programs, upstream just don't care. It's not just about odd architectures - they just don't care about NetBSD support, or support for different, alternative OSes. Perhaps that's fine, but consider this - the patches already exist and are tested, and they're maintained in pkgsrc. There are times people become indignant about being told how to fix their software for use under systems other than Windows, common Linux and macOS, on CPUs other than amd64 and aarch64. Yes, indignant. Some Python folks don't want to consider anyone would want to use Python on systems (like embedded) that don't have full IEEE floating point emulation. postgresql would rather mark platforms as broken than just let them compile the standard c portions of their code. People like to say, "they don't want to spend the time and energy", as if it adds work. No - there are examples where people choose the broken path when the not broken path is just as easy if not easier. Nobody is obligated to add or patch anything, even if the patches are freely given and well tested. But the fact that some people actively fight against portability is disturbing, and should be viewed with suspicion. The author of the article seems to have forgotten some history. They mention downsides of the c ecosystem, like the lack of standardized ways to build, the lack of consistent package management, and so on. But in every instance where those things have been imposed, the imposition has been problematic, has it not? Have we not seen security issues with PyPi? Have we not seen the dependency hell which is Ruby? The point is that nobody can tell anybody else how to do open source, so we'll always have a hundred different ways to do things. We also can't help that some people will be gatekeepers. But we personally can ignore the gatekeepers and help make things portable. Making excuses for why portability is somehow extra work is only encouraging gatekeeping. We don't need to do that - gatekeepers already have enough energy on their own.

pornel 4y ago

Are there architectures which LLVM doesn't support, which aren't retrocomputing or 16-bit microcontrollers?

Denvercoder9 4y ago
One example is the Xtensa architecture, used in the 32-bit ESP8266/ESP32 microcontrollers that are fairly popular with hobbyists (though Espressif seems to be moving to RISC-V for their newer offerings).
- pornel 4y ago
  Yeah, the RISC-V version seems to be supported: https://lib.rs/crates/esp8266 https://lib.rs/crates/esp8266
  - Denvercoder9 4y ago
    That crate requires a fork of the Rust compiler to use.
monocasa 4y ago
Mainline LLVM still doesn't support Xtensa or ARC AFAIK. ESP32 was Xtensa until their recent RISC-V based designs.

PaulHoule 4y ago

Personally C for Arduino programming drives me up the wall. With 32 general purpose registers in assembly language you can often keep all the variables in your inner loop in registers and still have a few left over for the interrupt handler. In C on the other hand you know it is moving the stack pointer around meaninglessly just so it can support recursive functions which aren’t really appropriate for embedded systems. I still write C for it because as beautiful as AVR-8 is, it is a dead end and if I need more capability I can take my C program to an ARM and maybe someday a RISC-V board. (I dream of embedding a soft AVR-8 on an FPGA with some special logic but the engineering students I know tell me that the Verilog class was like getting mauled by a bear.). At least gcc meets me halfway and has 24 bit ints in AVR-8. For that matter I just got a VisionFive 2 board that I need to put in a case and bring up with Linux. It has been a long time since I had to mess with other peoples C programs and bring them up on a new architecture but I think I’ll be doing it again.

phkahler 4y ago

Yeah why should we even care about s390 for some things? https://github.com/solvespace/solvespace/issues/1264 https://github.com/solvespace/solvespace/issues/1264 I don't think big commercial customers are designing airplanes with it.

cesarb 4y ago
That github issue is about the 64-bit s390x. The "weird architecture" being talked about here is the older 31-bit s390.

Karellen 4y ago

> Imagine, for a moment, that you’re a maintainer of a popular project. [...] You’ve also got a CI/CD pipeline that produces canonical releases of your project on tested architectures; [...] Because your project is popular, others also distribute it: Linux distributions, third-party package managers, and corporations seeking to deploy their own controlled builds. > You don’t know about any of the above until the bug reports start rolling in: users will report bugs that have already been fixed, bugs that you explicitly document as caused by unsupported configurations, bugs that don’t make any sense whatsoever. If 3rd-party build recipients are sending their bugs directly to you, that's a failure of the 3rd-party builders to take responsibility for their packages. They should be telling you to submit your bugs to them, so they can check their packaging, and then the packagers should talk to upstream only if there are issues to be resolved there. > You struggle to debug your users’ reports, since you don’t have access to the niche hardware, environments, or corporate systems that they’re running on. Yes, C supports lots of environments, and Rust supports quite a few (and hopefully more, soon), but I think it's perfectly fine for an upstream author to only support a subset of those. The benefits of Free Software is that if you want to get some software running on platforms the original author doesn't support but the language does, you can do that. Investigate the bug yourself. Figure out if it's in the app, or a 3rd-party library, or even the toolchain. Submit a patch. A good proportion of authors will happily accept patches for systems they themselves can't test on, if it's not too intrusive, and doesn't cause regressions on the platforms the author does support. They might be willing to entertain an intrusive patch series that allows for better cross-arch support, if you're willing to work with them on that. And if an author is not interested in helping you scratch your itch (ew! :-), create a fork. That should be easier than ever these days with the version control tools we have now, so much more than when Free Software was first envisioned. The author may even be willing to point other people who want to use their software on your platform your way (e.g. in their README), if only to get those users of their back!

slondr 4y ago
> The benefits of Free Software is that if you want to get some software running on platforms the original author doesn't support but the language does, you can do that. The exact cause of this entire controversy is that the software in question is switching to a language that supports less platforms.

ergonaught 4y ago

> As someone who likes C: this is all C’s fault. Really. I think I agree with almost everything in this post except this sentiment. Blaming the tool is silly. You may as well blame assembly language, and if you're doing that you may as well blame CPUs and electricity and you know what, physics is harmful, we should rewrite it in Rust.

celeritascelery 4y ago
If C had a standard way to test, build, and distribute packages then may of the authors concerns would be resolved. But it doesn’t, and other languages do.

jameshart 4y ago

There’s always been a weird double standard in open source software where it has been seen as ‘reasonable’ to expect the same code to compile and work on everything from an Arduino to an IBM mainframe, PDP11 to a RISC-V. ..But getting it to work on Windows? What? Do you expect the developers to fork out for Windows licenses? Don’t be ridiculous. Heck, some software is still downright snooty about the absurd idea that it should run cleanly on a Mac.

cesarb 4y ago
Playing devil's advocate: since the operating system mostly abstracts the underlying hardware, when programming with a high-level language, an IBM mainframe running Linux and a MIPS router running Linux are both closer to an AMD64 desktop running Linux than the same AMD64 desktop running Windows. And Windows is a particularly annoying case, since its native API is so different from the Linux native API (and details leak even when using API wrappers, for instance removing a recently-created file can fail on Windows because some other process like an antivirus grabbed a handle to it). But yes, I agree that a nontrivial fraction of the objections to making software run on Windows (or Mac) is for ideological reasons.
otabdeveloper4 4y ago
Microsoft has too many billions of dollars to count. Maybe they should be the ones to be held accountable for upholding community standards, not the free volunteers.
- jameshart 4y ago
  Which ‘community standards’ do you mean? POSIX?
  - otabdeveloper4 4y ago
    POSIX is a corporate standard. Community standards are usually not formalized.
pohl 4y ago

docandrew 4y ago

I think the frustrating thing is that, while rust may consider one of these architectures “tier 2” or unsupported or whatever, there’s a Python interpreter that presumably doesn’t, and whoever is using the Python cryptography package expects it to work the same place all their other code does. And very likely it’s some dependency several layers deep. If my code is going to end up architecture-dependent then why bother with an interpreted language in the first place?

StreamBright 4y ago
> why bother with an interpreted language in the first place +1 As a person who uses Python for 12 years professionally I usually try to avoid it as much as I can. When you have a Python problem you usually have a C, C++, Rust, libc, arch problem that you do not realise. Most of the useful parts of Python are written in C, C++, Fortran, Rust so when you try to deploy it some less frequently used platform it can burst into flames the worst kind of ways. You can try to deploy the AWS Lambda / Python 3.9 and see the lolz. I have spent more hours on trying to get some Python lib work on a platform than learning Rust. I think interpreted languages are a dead end especially with bad practices. I make my living writing Python but it is a misery every direction. It is not an accident that Rust is the most loved language continuously because it just works. Anything I try to do in it works as expected and I am not walking on a mine field. Let me give you a simple example how Python can surprise you. Lets create an app in Python that uses a lib called X. You build your project and everything works locally, unit tests are ok, integration tests too and so on. Now deploy this code to AWS Lambda (not your choice, employer decided to go with that). You package everything up on a Linux that matches the architecture of the target (lets say X64). If you are not familiar with setting the target Python version with pip (and most documentation does not mention that for Lambda) you deploy your code and try to invoke it. Fail. You have X.311.so in your deployment package and Lambda tries to load X.39.so. Now you need to figure out how to set the version or have a build env that matches the CPU AND the Python version with the target system. I could continue this rabbit hole for some more but the point is that you can't talk about Python alone, you need to pull in the Cartesian product of libc, libXX, C, C++, Fortrant, all the compilers for these, cpu architectures and Python versions. On a lucky they you might have a working system. With Rust everything just worked the first time we tried to use it. I could not believe it how easy it was to put out a working system at the first try. It only beats Python by an order of magnitude in terms of performance but the amount of effort it took us to deploy it was also much less.

pyuser583 4y ago

If only we had stuck with Ada.

rbanffy 4y ago

> Give up on weird ISAs and platforms NEVER!

Hackbraten 4y ago
I didn’t interpret it as “say goodbye to niche arch support entirely” but rather as “it’s about time we stop externalizing to the OSS volunteer community the huge burden of supporting weird platforms, let’s rather shift that labor and cost onto the actual stakeholders of those architectures.” That doesn’t imply end users of niche architectures are supposed to lose their favorite apps.
- cbmuser 4y ago
  If you look at the market shares, you’ll quickly realize that this would mean abandoning all the BSDs, Haiku, Hurd, all the obscure Linux distributions and so on. Just support macOS, RHEL/SLES, Debian, Ubuntu and Windows. Everything else gets tagged with “UNSUPPORTED/WONTFIX”. But please don’t complain when your favorite operating system is not supported. Or, you know, we could just stop trying to tell others what targets to use. And, FWIW, Rust is actually getting support for more architectures thanks to the GCC codegen and GCC frontend.
  - dwheeler 4y ago
    Why so many architectures? Ensure your software can only run on Windows: https://gs.statcounter.com/os-market-share/desktop/worldwide https://gs.statcounter.com/os-market-share/desktop/worldwide :-). Vive la difference.
  - sophacles 4y ago
    You're welcome to do it that way for your software. Others are allowed to choose some subset or superset of your choice. If they don't like your choice, they are allowed to fork the software to support what they want. Case in point - the gcc projects for rust are not officially supported by rust. They are alternative rust compilers - the authors of those have explicitly stated they will follow the features (etc) of the official rustc, and maybe the rust language team will consider those other compilers when designing new features, but it's not on the rust language team to do so. Another case in point - many projects don't support most linux distros. The job of the linux distro is maintain a fork (usually in the form of patches against upstream in the source package) of the software that works well with their system.

zamalek 4y ago

> The security of a program is a function of its own design and testing, as well as the design, testing, and basic correctness of its underlying platform: everything from the userspace, to the kernel, to the compilers themselves. You don't even have to worry about bugs. There are many cryptography primitives that require constant-time to guarantee security. That means that the latency of each machine code instruction could be vital to security. The latency of instructions between vendors within the same architecture is already a concern.

woodruffw 4y ago
This is a good point! But to be maximally precise: variations between each ISA implementation's latencies is not particularly important; what's important is that instructions on data-dependent paths are not themselves data-dependent in timing. In other words: it's okay if AMD64 takes 9 cycles to execute an instruction while IA32e takes 7; the problem arises when AMD64 takes 9 + k cycles for k bytes of input while cryptographic engineers have tested on IA32e and assumed a constant overhead of 7 cycles.
- Gordonjcp 4y ago
  > the problem arises when AMD64 takes 9 + k cycles for k bytes of input while cryptographic engineers have tested on IA32e and assumed a constant overhead of 7 cycles. Why is that a problem? Doesn't that just leak something you could determine by just looking at the CPU type?
  - bluGill 4y ago
    That also leaks how many bytes of data are being processed. This sometimes matters. When checking for a password match you have to check all characters in the.string otherwise you leak where the mismatch was. Even in a properly salted and hashed scheme that makes breaking the password easier.
  - jeroenhd 4y ago
    These side channel attacks allow perfectly secure algorithms to leak plaintext or even complete keys.