11 ms·

Why do programming language maintainers for a front end to LLVM (rust) need to consider/patch/modify their front end to properly support a llvm backend, like AV

Why do programming language maintainers for a front end to LLVM (rust) need to consider/patch/modify their front end to properly support a llvm backend, like AVR? (My mental model was that once you add a backend to Llvm, every programming language that uses llvm should magically work with it.) Does LLVM's design fail to capture the full problem space? Are there out of band concerns that do not lend themselves to be generally expressed?

JoshTriplett 10y ago
Quoting the article: > Rust's version of LLVM has a few patches on top of stock LLVM, and it also tended to have extra commits cherry-picked on top. Apart from that, Rust needs very little per-architecture glue code. And someone needs to run the Rust testsuite on the resulting version of Rust and make sure it passes.
elihu 10y ago
From the blog post, it sounds like the problem is that Rust doesn't use standard LLVM, they maintain a separate branch with a bunch of patches that are specific to Rust. So, adding AVR support to Rust means adding AVR support to Rust's branch of LLVM, which is a hassle. Rust is about to move to a newer version of LLVM, though, so when that happens they'll be pulling in the AVR support that's already in mainline LLVM.
dbaupp 10y ago
LLVM isn't completely platform independent (e.g. all integers are fixed-width, so the frontend needs to know the pointer size to emit pointer-sized integers) and there are some configuration options to be passed down to LLVM/the linker. Additionally, it is useful for the front-end to understand things like the layout of types, and the standard library usually also needs changes to paper over any platform differences (although I imagine AVR doesn't need many, mostly in the compiler-support library compiler-rt, given not much of the standard library will work).
hedgehog 10y ago
https://internals.rust-lang.org/t/adding-16-bit-pointer-support/2484 https://internals.rust-lang.org/t/adding-16-bit-pointer-supp...
RX14 10y ago
If you want an example of a llvm-based programming language porting to a new architecture, see [1]. The issues has a checklist of things that were done, and a readable diff. Some of the things that needed to be implemented in the frontend was the ABI, unwind support, C bindings for the new architecture. Some of these don't apply as much to rust but you can see that LLVM doesn't do a perfect job. The diff is still quite small however. [1] https://github.com/crystal-lang/crystal/pull/3491 https://github.com/crystal-lang/crystal/pull/3491
kps 10y ago
Going to express this in C terms since that language has a longer history in the space. For little processors like AVR, and bigger but specialized ones (e.g. DSPs), standard ANSI C is neither necessary nor sufficient. For instance, nobody is ever going to do IEEE754 floating point arithmetic on an AVR, or on a DSP56000. But these processors have other properties that a programmer is likely to want to use that are not expressible in standard C (sometimes, not even with intrinsics); for the most common cases, C has the TR18037 extensions¹. In the AVR case, on the biggest chips a RAM address takes 2 bytes while a flash address is 3 bytes, and on the smallest a RAM address is 1 byte while a flash address is 2, and of course the values overlap (flash-address-0xFE points to different memory than RAM-address-0xFE), and the indirection code is different. So how big is a plain 'char *', and how do you dereference it? On a small AVR you don't want to be wasting precious memory storing and copying 2 bytes when 1 will do, and you sure don't want a run-time test on every dereference — if you had the cost/time/battery for that you wouldn't have picked² a small AVR in the first place. So you need to be able to tell the compiler what kind of memory a pointer points to. The big free toolchains (gcc/llvm) don't pay much attention to non-mainstream processors, which is why a handful of commercial compilers still exist. I suppose (not being a Rust expert) that a Rust compiler could figure out that a non-mut 'static can go in flash, and track address spaces, and auto-genericize functions that take a reference… but I'll bet it doesn't, yet. ¹ http://www.open-std.org/JTC1/SC22/WG14/www/docs/n1169.pdf http://www.open-std.org/JTC1/SC22/WG14/www/docs/n1169.pdf ² no pun intended originally, but I'm leaving it and gratuitously calling attention to it.

JoshTriplett 10y ago

Quoting the article: > Rust's version of LLVM has a few patches on top of stock LLVM, and it also tended to have extra commits cherry-picked on top. Apart from that, Rust needs very little per-architecture glue code. And someone needs to run the Rust testsuite on the resulting version of Rust and make sure it passes.

elihu 10y ago

From the blog post, it sounds like the problem is that Rust doesn't use standard LLVM, they maintain a separate branch with a bunch of patches that are specific to Rust. So, adding AVR support to Rust means adding AVR support to Rust's branch of LLVM, which is a hassle. Rust is about to move to a newer version of LLVM, though, so when that happens they'll be pulling in the AVR support that's already in mainline LLVM.

dbaupp 10y ago

LLVM isn't completely platform independent (e.g. all integers are fixed-width, so the frontend needs to know the pointer size to emit pointer-sized integers) and there are some configuration options to be passed down to LLVM/the linker. Additionally, it is useful for the front-end to understand things like the layout of types, and the standard library usually also needs changes to paper over any platform differences (although I imagine AVR doesn't need many, mostly in the compiler-support library compiler-rt, given not much of the standard library will work).

hedgehog 10y ago

https://internals.rust-lang.org/t/adding-16-bit-pointer-support/2484 https://internals.rust-lang.org/t/adding-16-bit-pointer-supp...

RX14 10y ago

If you want an example of a llvm-based programming language porting to a new architecture, see [1]. The issues has a checklist of things that were done, and a readable diff. Some of the things that needed to be implemented in the frontend was the ABI, unwind support, C bindings for the new architecture. Some of these don't apply as much to rust but you can see that LLVM doesn't do a perfect job. The diff is still quite small however. [1] https://github.com/crystal-lang/crystal/pull/3491 https://github.com/crystal-lang/crystal/pull/3491

kps 10y ago

Going to express this in C terms since that language has a longer history in the space. For little processors like AVR, and bigger but specialized ones (e.g. DSPs), standard ANSI C is neither necessary nor sufficient. For instance, nobody is ever going to do IEEE754 floating point arithmetic on an AVR, or on a DSP56000. But these processors have other properties that a programmer is likely to want to use that are not expressible in standard C (sometimes, not even with intrinsics); for the most common cases, C has the TR18037 extensions¹. In the AVR case, on the biggest chips a RAM address takes 2 bytes while a flash address is 3 bytes, and on the smallest a RAM address is 1 byte while a flash address is 2, and of course the values overlap (flash-address-0xFE points to different memory than RAM-address-0xFE), and the indirection code is different. So how big is a plain 'char *', and how do you dereference it? On a small AVR you don't want to be wasting precious memory storing and copying 2 bytes when 1 will do, and you sure don't want a run-time test on every dereference — if you had the cost/time/battery for that you wouldn't have picked² a small AVR in the first place. So you need to be able to tell the compiler what kind of memory a pointer points to. The big free toolchains (gcc/llvm) don't pay much attention to non-mainstream processors, which is why a handful of commercial compilers still exist. I suppose (not being a Rust expert) that a Rust compiler could figure out that a non-mut 'static can go in flash, and track address spaces, and auto-genericize functions that take a reference… but I'll bet it doesn't, yet. ¹ http://www.open-std.org/JTC1/SC22/WG14/www/docs/n1169.pdf http://www.open-std.org/JTC1/SC22/WG14/www/docs/n1169.pdf ² no pun intended originally, but I'm leaving it and gratuitously calling attention to it.