5 ms·

I'm waiting for Rust to support both static and dynamical link equally well. static link produces binaries too large for my embedded boards when stdlib is neede

I'm waiting for Rust to support both static and dynamical link equally well. static link produces binaries too large for my embedded boards when stdlib is needed. dynamic link looks like a second class citizen to me in Rust. Is there some hope?

slavapestov 4y ago
Since Rust doesn’t have a stable ABI or separately compiled generics, wouldn’t dylibs necessarily be limited to exporting a C ABI?
- woodruffw 4y ago
  My understanding (which might be wrong) is that you can rely on Rust not making incompatible ABI decisions between builds of the same optimization level on the same compiler version. In other words: you don't need to export a C ABI as long as you're able to enforce that every crate is built dynamically with the same compiler and flags. That is of course still a significant restriction!
  - NonNefarious 4y ago
    ABI shenanigans are what killed the whole "off-the-shelf components" promise of C++ and object orientation. They really made the same mistake again?
    - woodruffw 4y ago
      I don't know how you arrived at that. C++ has the exact opposite ABI problems: it has a far-too-stable ABI that leaks thanks to poor abstractions (header files and template expansion in first-party code). Institutional C++ users have far too many implicit ABI dependencies to ever sign off on breakages, so C++ stdlib maintainers are unable to optimize much of the `std` namespace. This in contrast to Rust, which actually does fulfill the "off-the-shelf" promise for most use cases. It's really refreshing to be able to do `cargo build` on nearly any host and get a single binary. Edit: Jason Turner covers C++'s ABI woes excellently here: https://www.youtube.com/watch?v=By7b19YIv8Q https://www.youtube.com/watch?v=By7b19YIv8Q
      - jcelerier 4y ago
        C++ has Schrödinger's ABI: some will say that it has a too stable ABI and others will say that its ABI is not stable at all. Which one is it ? https://stackoverflow.com/questions/67839008/please-explain-the-c-abi https://stackoverflow.com/questions/67839008/please-explain-... Can one link two rust static libraries communicating with actual rust symbols (not just C ones), one built with gcc-rs (assuming that this is possible yet), the other built with rustc with its default x86_64-pc-windows-msvc abi ? if not, it's at the exact same state ABI-wise than C++
        woodruffw 4y ago
        Well, it's a multiparadigm language, so both :-) I'm being facetious, but it really is both: it's too unstable in ways that are bad, and also too stable in other ways that are bad. Edit: since you updated your comment: Rust is not in the same position as C++. C++ has an unstable ABI that it cannot break; Rust has an unstable ABI that it can (and regularly does break). Rust works around this with developer tooling; C++ largely ignores the problem and requires developers to learn the secret rules through blood and tears.
        jcelerier 4y ago
        > Rust works around this with developer tooling; how does that work for companies that may use proprietary rust libraries? the proprietary lib authors won't recompile their shit for you just because you upgraded to the latest rustc
        woodruffw 4y ago
        As pointed out, the C ABI is stable. If companies absolutely feel the need to distribute Rust shared libraries and cannot guarantee the compiler being used, then it's a reasonable choice. I suspect that isn't very common, however. More realistically, the answer is to change the interface: Rust is pretty popular in software architectures where the unit of operation is a networked or IPC'd service; vendors can distribute binaries that communicate at that layer instead.
    - sanxiyn 4y ago
      Rust wants to fix this, but it's an extremely hard problem and other tasks received higher priority. See "How Swift Achieved Dynamic Linking Where Rust Couldn't" for details: https://faultlore.com/blah/swift-abi/ https://faultlore.com/blah/swift-abi/
      - NonNefarious 4y ago
        Thanks for that referral. On a side note: I like Swift for the most part, but the last time I looked at upcoming language features they seemed like a kitchen-sink approach that's bloating the language and would better have been left to libraries. I've read this sentiment in more than one place, too.
woodruffw 4y ago
The workflow with `cargo` isn't as nice with dynamic linking, but `crate-type = "dylib"` has worked for a while now. Any crate built with that should produce a shared object instead, which you can link in like any other.
absolutelynobo 4y ago
you can use no_std when available. not sure how dynamic linking would help when your boards don’t have enough (flash ?) space… the linked library still needs to go somewhere? how small of a flash medium are you using?
- ReactiveJelly 4y ago
  Maybe multiple entrypoints? The busybox style of packing everything into subcommands of one big exe is admittedly a hack, but... if it works for Busybox, and it works for Go, and it works for Git, and it works for Docker, then it works for me.
- synergy20 4y ago
  yes no_std for kernel or boot code or MCU boards, that's what rust-embedded is doing and I think it's fine. many embedded boards typically have 16~64MB Flash running Linux with musl, one rust binary statically linked can easily exceeding 10MB. multi-entry is hard to manage when you have quite a few unrelated tasks, so yes I really need a true shared lib based rust for the mid-range embedded boards, which are, quite a huge number.
  - fleventynine 4y ago
    I'm not sure what you mean by multi-entry, but it's hard to beat a busybox-style binary for code-size, especially with LTO. If you're building everything into a single filesystem image anyways, this is almost always smaller than dynamic linking.
raggi 4y ago
dylib linking works well enough particularly for embedded cases. As an example Fuchsia did some research on this comparing it to C++. dynamically linking libstd amortized similar savings to dynamically linking libstd++, with trivial binaries going from hundreds of kilobytes to tens of kilobytes with a megabyte sized shared object for std. As you'll read elsewhere in the thread there isn't an ABI guarantee, as there also isn't so much for C++ either - but particularly for embedded you can solve this in your build process. A competent package manager in a distribution can also solve for this.
schemescape 4y ago
This is a tangent, but how do people comply with the MIT (or other) license when statically linking binaries? You have to include the copyright notice and license, but I almost never see that being done (and, based on the Rust projects I’ve seen, there are often tens, or even hundreds of MIT/BSD/Apache licensed dependencies).
- deadbeeves 4y ago
  People who use permissive licenses usually don't care about the license-inclusion clause, so most people just don't bother with it until someone comes asking. At the end of the day, fixing that problem is just a matter of including a new text file in the next release. It's nowhere near as big of a deal as when someone complains that you're not complying with the GPL. By the way, whether you're linking statically or dynamically doesn't affect whether you ought to include the licenses of your dependencies. At least not for BSD, I'm less certain for MIT.
  - CameronNemo 4y ago
    I disagree with both paragraphs. By choosing a license that requires attribution, developers are communicating a preference. MIT0 and 0BSD exist. If someone does not care about attribution, they would use one of those licenses. Dynamically linking to a MIT lib is not the same as distributing an MIT lib, so yeah static linking to the lib is completely different, according to the terms of the license. If you are working professionally as a software developer, or even just appreciate open source contributions, please take licensing terms more seriously. Kind regards.
    - c0mptonFP 4y ago
      That's one of the thing that riles me up. The vast majority of programmers, especially in the open source community, don't take licensing seriously.
    - deadbeeves 4y ago
      They certainly don't care enough to enforce it. >Dynamically linking to a MIT lib is not the same as distributing an MIT lib Would you mind citing the clause that makes that distinction?

slavapestov 4y ago

Since Rust doesn’t have a stable ABI or separately compiled generics, wouldn’t dylibs necessarily be limited to exporting a C ABI?

woodruffw 4y ago
My understanding (which might be wrong) is that you can rely on Rust not making incompatible ABI decisions between builds of the same optimization level on the same compiler version. In other words: you don't need to export a C ABI as long as you're able to enforce that every crate is built dynamically with the same compiler and flags. That is of course still a significant restriction!
- NonNefarious 4y ago
  ABI shenanigans are what killed the whole "off-the-shelf components" promise of C++ and object orientation. They really made the same mistake again?
  - woodruffw 4y ago
    I don't know how you arrived at that. C++ has the exact opposite ABI problems: it has a far-too-stable ABI that leaks thanks to poor abstractions (header files and template expansion in first-party code). Institutional C++ users have far too many implicit ABI dependencies to ever sign off on breakages, so C++ stdlib maintainers are unable to optimize much of the `std` namespace. This in contrast to Rust, which actually does fulfill the "off-the-shelf" promise for most use cases. It's really refreshing to be able to do `cargo build` on nearly any host and get a single binary. Edit: Jason Turner covers C++'s ABI woes excellently here: https://www.youtube.com/watch?v=By7b19YIv8Q https://www.youtube.com/watch?v=By7b19YIv8Q
    - jcelerier 4y ago
      C++ has Schrödinger's ABI: some will say that it has a too stable ABI and others will say that its ABI is not stable at all. Which one is it ? https://stackoverflow.com/questions/67839008/please-explain-the-c-abi https://stackoverflow.com/questions/67839008/please-explain-... Can one link two rust static libraries communicating with actual rust symbols (not just C ones), one built with gcc-rs (assuming that this is possible yet), the other built with rustc with its default x86_64-pc-windows-msvc abi ? if not, it's at the exact same state ABI-wise than C++
      - woodruffw 4y ago
        Well, it's a multiparadigm language, so both :-) I'm being facetious, but it really is both: it's too unstable in ways that are bad, and also too stable in other ways that are bad. Edit: since you updated your comment: Rust is not in the same position as C++. C++ has an unstable ABI that it cannot break; Rust has an unstable ABI that it can (and regularly does break). Rust works around this with developer tooling; C++ largely ignores the problem and requires developers to learn the secret rules through blood and tears.
        jcelerier 4y ago
        > Rust works around this with developer tooling; how does that work for companies that may use proprietary rust libraries? the proprietary lib authors won't recompile their shit for you just because you upgraded to the latest rustc
        woodruffw 4y ago
        As pointed out, the C ABI is stable. If companies absolutely feel the need to distribute Rust shared libraries and cannot guarantee the compiler being used, then it's a reasonable choice. I suspect that isn't very common, however. More realistically, the answer is to change the interface: Rust is pretty popular in software architectures where the unit of operation is a networked or IPC'd service; vendors can distribute binaries that communicate at that layer instead.
  - sanxiyn 4y ago
    Rust wants to fix this, but it's an extremely hard problem and other tasks received higher priority. See "How Swift Achieved Dynamic Linking Where Rust Couldn't" for details: https://faultlore.com/blah/swift-abi/ https://faultlore.com/blah/swift-abi/
    - NonNefarious 4y ago
      Thanks for that referral. On a side note: I like Swift for the most part, but the last time I looked at upcoming language features they seemed like a kitchen-sink approach that's bloating the language and would better have been left to libraries. I've read this sentiment in more than one place, too.

woodruffw 4y ago

The workflow with `cargo` isn't as nice with dynamic linking, but `crate-type = "dylib"` has worked for a while now. Any crate built with that should produce a shared object instead, which you can link in like any other.

absolutelynobo 4y ago

you can use no_std when available. not sure how dynamic linking would help when your boards don’t have enough (flash ?) space… the linked library still needs to go somewhere? how small of a flash medium are you using?

ReactiveJelly 4y ago
Maybe multiple entrypoints? The busybox style of packing everything into subcommands of one big exe is admittedly a hack, but... if it works for Busybox, and it works for Go, and it works for Git, and it works for Docker, then it works for me.
synergy20 4y ago
yes no_std for kernel or boot code or MCU boards, that's what rust-embedded is doing and I think it's fine. many embedded boards typically have 16~64MB Flash running Linux with musl, one rust binary statically linked can easily exceeding 10MB. multi-entry is hard to manage when you have quite a few unrelated tasks, so yes I really need a true shared lib based rust for the mid-range embedded boards, which are, quite a huge number.
- fleventynine 4y ago
  I'm not sure what you mean by multi-entry, but it's hard to beat a busybox-style binary for code-size, especially with LTO. If you're building everything into a single filesystem image anyways, this is almost always smaller than dynamic linking.

raggi 4y ago

dylib linking works well enough particularly for embedded cases. As an example Fuchsia did some research on this comparing it to C++. dynamically linking libstd amortized similar savings to dynamically linking libstd++, with trivial binaries going from hundreds of kilobytes to tens of kilobytes with a megabyte sized shared object for std. As you'll read elsewhere in the thread there isn't an ABI guarantee, as there also isn't so much for C++ either - but particularly for embedded you can solve this in your build process. A competent package manager in a distribution can also solve for this.

schemescape 4y ago

This is a tangent, but how do people comply with the MIT (or other) license when statically linking binaries? You have to include the copyright notice and license, but I almost never see that being done (and, based on the Rust projects I’ve seen, there are often tens, or even hundreds of MIT/BSD/Apache licensed dependencies).

deadbeeves 4y ago
People who use permissive licenses usually don't care about the license-inclusion clause, so most people just don't bother with it until someone comes asking. At the end of the day, fixing that problem is just a matter of including a new text file in the next release. It's nowhere near as big of a deal as when someone complains that you're not complying with the GPL. By the way, whether you're linking statically or dynamically doesn't affect whether you ought to include the licenses of your dependencies. At least not for BSD, I'm less certain for MIT.
- CameronNemo 4y ago
  I disagree with both paragraphs. By choosing a license that requires attribution, developers are communicating a preference. MIT0 and 0BSD exist. If someone does not care about attribution, they would use one of those licenses. Dynamically linking to a MIT lib is not the same as distributing an MIT lib, so yeah static linking to the lib is completely different, according to the terms of the license. If you are working professionally as a software developer, or even just appreciate open source contributions, please take licensing terms more seriously. Kind regards.
  - c0mptonFP 4y ago
    That's one of the thing that riles me up. The vast majority of programmers, especially in the open source community, don't take licensing seriously.
  - deadbeeves 4y ago
    They certainly don't care enough to enforce it. >Dynamically linking to a MIT lib is not the same as distributing an MIT lib Would you mind citing the clause that makes that distinction?