5 ms·

The best thing about Cargo is not that it's the best build system (feature-wise it's rather basic), but that there's nothing else for Rust. Projects never have

by pornel 6y ago

The best thing about Cargo is not that it's the best build system (feature-wise it's rather basic), but that there's nothing else for Rust. Projects never have to argue whether Cargo is better than Mergo or Ronan or Rustfiles or their grandad's bash script. All the tooling knows how to build any project. Every editor knows where to look for files. https://docs.rs https://docs.rs can generate documentation for 40000 different projects using the same build command.

jnxx 6y ago
What's also good is that one can combine cargo with Guix as an OS/multi-language package manager, for example when other tools or C libraries (say, alsa-lib) are needed. Guix provides currently 15,000 packages including many rust packages (https://guix.gnu.org/packages/R/page/21/ https://guix.gnu.org/packages/R/page/21/ ), and uses of course cargo for building rust packages. The reason why I think this is the right way to do it is because in big , quickly-moving projects, the only comfortable way out of dependency hell is to use a Linux distribution with checked-to-work dependencies, or one is damned to the task of building a distribution oneself, and a checked distribution of packages is precisely what Guix provides. Because own, or project-specific package recipes, which contain dependency and build instructions, can just be added to the public package definitions (https://guix.gnu.org/blog/2018/a-packaging-tutorial-for-guix/ https://guix.gnu.org/blog/2018/a-packaging-tutorial-for-guix...), this allows to deterministically and hermetically build multi-language projects on top of a very solid base. And for testing with another version of a C dependency (say, alsa-lib 1.1.9), one just needs a branch with another version number in the build recipe.
- choeger 6y ago
  That sounds about fantastic. Is there a non-root way to build and a way to install at least some of the packages in my $HOME? Because as a developer I need to work with different branches. edit: How does it handle python wheels and other non-source language-specific packages?
  - jnxx 6y ago
    > Is there a non-root way to build and a way to install at least some of the packages in my $HOME Yes. You can install Guix as a package manager on top of your POSIX OS. Then, you have profiles, and you can install packages into your personal profile in your $HOME, or into temporary project-specific environments. You can also define the environment by writing a manifest file which list requested packages and their versions, and put that file under version control. How it is used is documented here: https://guix.gnu.org/manual/en/guix.html#Package-Management https://guix.gnu.org/manual/en/guix.html#Package-Management > How does it handle python wheels and other non-source language-specific packages? Of course you can install Python packages: https://guix.gnu.org/packages/P/page/12/ https://guix.gnu.org/packages/P/page/12/ If there are no pre-built packages for your machine architecture (say, you are developing on ARM), they are built and installed on your machine (or, a build server which you defined), otherwise you get automatically a compiled package from the Guix server. Also, and in difference to pip and so on, any needed C libraries will be installed as well, with the right versions. So, it is a bit like Conda, but it will also work if your project happens to mix, say, Python, C, C++ and Rust, and perhaps runs on mips or armhf. (My next side project will be a Game with a strategy engine in Rust, and a GUI written in Racket, just as an example). I have not yet tried it with Python, but I can confirm it works with Common Lisp libraries. Basically, you can get anything which is FLOSS, and if you need proprietary stuff you might be able it via a non-free package channel (similar to Debian).
IshKebab 6y ago
Not exactly true. Google uses Bazel for Rust code. Also there are definitely nice features of Cargo compared to other build systems, especially C build systems. But I think you're mostly right. Fragmenting the Rust build ecosystem would be quite annoying.
- ivanbakel 6y ago
  Google uses pure Bazel, rather than just a Cargo wrapper? I was under the impression that Bazel was for plugging build and dependency systems into - that's certainly how I've seen it used.
  - wocram 6y ago
    Bazel is generally much more useful when it can manage the entire build. https://github.com/bazelbuild/rules_rust https://github.com/bazelbuild/rules_rust implements this logic for rust. It calls rustc directly, without cargo for the build, and interacts with cargo through a shim to fetch/vendor dependencies. Google uses various non-cargo build systems to build rust. I don't think Google is actually using bazel for this internally, eg. Fuchsia uses https://fuchsia.googlesource.com/fuchsia/+/master/third_party/rust_crates/ https://fuchsia.googlesource.com/fuchsia/+/master/third_part...
- wocram 6y ago
  What makes you think Google is using bazel for rust? I haven't seen or heard any evidence of this.
  - gravypod 6y ago
    Internally at Google the vendor all source code and make it build using bazel. These rules also show some of the open source versions of this work: https://github.com/bazelbuild/rules_rust https://github.com/bazelbuild/rules_rust As well as the features that bazel provides you. The caching, packaging, code gen, cross-language deps/interfacing, and other magic that make bazel extremely amazing to use.
suprfnk 6y ago
Is this not also a function of being at least a /good/_ build system? If Cargo were too basic, or too hard to use, there would definitely be people creating a better one. This then causes the fragmentation you are talking about.
- steveklabnik 6y ago
  Yes, this matters too. I've been bumping into some tough limitations of Cargo lately, but it's still good enough to stick with it, and paper over them, rather than throw the whole thing out wholesale.
  - mholub 6y ago
    what kind of limitations in a few words?
    - steveklabnik 6y ago
      https://users.rust-lang.org/t/complex-build-advice/48779 https://users.rust-lang.org/t/complex-build-advice/48779 is the best I can say at the moment.
pjmlp 6y ago
Not being able to deal with binary libraries and multiple languages is a big pain point for me, and no build.rs is not a solution, rather a workaround.
- jnxx 6y ago
  If it is a common package, you could use Guix to retrieve binary library packages for your architecture. (It will build from source if there is no pre-built package).
  - pjmlp 6y ago
    Guix is a Linux only solution, and the whole point is not to build anything from 3rd party dependencies.
    - jnxx 6y ago
      > Guix is a Linux only solution, and the whole point is not to build anything from 3rd party dependencies. First, GNU Guix requires a POSIX system, and it will happily run e.g. on top of Debian in a suitable VM. (It will also run as a stand-alone OS which is nice if you want a bootstrapped, totally deterministic system - not good for gaming or casual development yet but there are serious long-term applications for that.) Also, as said it will build from source only if there is no pre-built binary package available for your platform, otherwise it will use cached binaries. That's, for example, a great use if you need to use cargo with a complex multi-architecture set-up. Just don't forgot to add plenty of disk space - here is a good use for Terabytes-size disks. Further, you can also set up your own build server, which caches binaries. The decision that it depends on the source code being available is rooted in the philosophy of the project; Guix is a GNU project and is intended to support Free software, as in FLOSS. I think this is also technically a good decision, because if the source code is not available, this leads sooner or later to practically unsolvable problems; think in all the expensive lab equipment in science which runs on decades old, obsolete, unsupported, unsafe platforms just because that expensive hardware requires a binary driver which won't run on more modern platforms. Or think in your scanner that stopped working after the last OS upgrade because the scanner vendor thought you ought buy a new one; that scanner will probably work fine under GNU/Linux, just as an illustration to the point being made. The need to make software reproducible is especially important in science where the scientific method depends on that experiments, and their evaluation can be repeated, but there is just no budget to port everything to the newest Python3.x, or OpenGL Y, or Qt-Z, or Matlab-ω. Science is competitive, no less than business, and you don't make a career in science with porting old stuff. But back to "Linux only" - right, there are platforms that Guix will not work on, like Windows. But complaining about that seems, at best, extremely naïve to me: By anyone who has the slightest knowledge in the matter it is well known that these proprietary platforms have literally spent decades on engineering to lock users in into their proprietary solutions. (And I think that the situation especially with C++ is in part a result of that, there is no inevitable technical reason that C++ does not have a standard ABI, or standard library discovery, it was just made that way to stop competitors similar to Borland's Delphi.) Complaining that their OS (while claiming POSIX compatibility) does not support GNU Guix is justs as, perdon my french, just as silly as complaining that you can't fix the broken battery of your MacBook yourself by taking a screwdriver and replacing it with a standard part. You won't have a screwdriver that fits precisely because the MacBook's vendor thought that you are not supposed to fix it yourself. Of course, the marketing department will tell you that their platform is "compatible to open source" - I have a bridge to sell you. These systems are just not designed for that, and the inability to use Guix on them is a consequence of your decision to use such a system; you are the one who will pay the costs of that, by having an obsolete scanner, or, far more expensive, by having obsolete, unmaintainable software years from now which is too expensive to port to a new supporting API.

by pornel 6y ago

jnxx 6y ago

What's also good is that one can combine cargo with Guix as an OS/multi-language package manager, for example when other tools or C libraries (say, alsa-lib) are needed. Guix provides currently 15,000 packages including many rust packages (https://guix.gnu.org/packages/R/page/21/ https://guix.gnu.org/packages/R/page/21/ ), and uses of course cargo for building rust packages. The reason why I think this is the right way to do it is because in big , quickly-moving projects, the only comfortable way out of dependency hell is to use a Linux distribution with checked-to-work dependencies, or one is damned to the task of building a distribution oneself, and a checked distribution of packages is precisely what Guix provides. Because own, or project-specific package recipes, which contain dependency and build instructions, can just be added to the public package definitions (https://guix.gnu.org/blog/2018/a-packaging-tutorial-for-guix/ https://guix.gnu.org/blog/2018/a-packaging-tutorial-for-guix...), this allows to deterministically and hermetically build multi-language projects on top of a very solid base. And for testing with another version of a C dependency (say, alsa-lib 1.1.9), one just needs a branch with another version number in the build recipe.

choeger 6y ago
That sounds about fantastic. Is there a non-root way to build and a way to install at least some of the packages in my $HOME? Because as a developer I need to work with different branches. edit: How does it handle python wheels and other non-source language-specific packages?
- jnxx 6y ago
  > Is there a non-root way to build and a way to install at least some of the packages in my $HOME Yes. You can install Guix as a package manager on top of your POSIX OS. Then, you have profiles, and you can install packages into your personal profile in your $HOME, or into temporary project-specific environments. You can also define the environment by writing a manifest file which list requested packages and their versions, and put that file under version control. How it is used is documented here: https://guix.gnu.org/manual/en/guix.html#Package-Management https://guix.gnu.org/manual/en/guix.html#Package-Management > How does it handle python wheels and other non-source language-specific packages? Of course you can install Python packages: https://guix.gnu.org/packages/P/page/12/ https://guix.gnu.org/packages/P/page/12/ If there are no pre-built packages for your machine architecture (say, you are developing on ARM), they are built and installed on your machine (or, a build server which you defined), otherwise you get automatically a compiled package from the Guix server. Also, and in difference to pip and so on, any needed C libraries will be installed as well, with the right versions. So, it is a bit like Conda, but it will also work if your project happens to mix, say, Python, C, C++ and Rust, and perhaps runs on mips or armhf. (My next side project will be a Game with a strategy engine in Rust, and a GUI written in Racket, just as an example). I have not yet tried it with Python, but I can confirm it works with Common Lisp libraries. Basically, you can get anything which is FLOSS, and if you need proprietary stuff you might be able it via a non-free package channel (similar to Debian).

IshKebab 6y ago

Not exactly true. Google uses Bazel for Rust code. Also there are definitely nice features of Cargo compared to other build systems, especially C build systems. But I think you're mostly right. Fragmenting the Rust build ecosystem would be quite annoying.

ivanbakel 6y ago
Google uses pure Bazel, rather than just a Cargo wrapper? I was under the impression that Bazel was for plugging build and dependency systems into - that's certainly how I've seen it used.
- wocram 6y ago
  Bazel is generally much more useful when it can manage the entire build. https://github.com/bazelbuild/rules_rust https://github.com/bazelbuild/rules_rust implements this logic for rust. It calls rustc directly, without cargo for the build, and interacts with cargo through a shim to fetch/vendor dependencies. Google uses various non-cargo build systems to build rust. I don't think Google is actually using bazel for this internally, eg. Fuchsia uses https://fuchsia.googlesource.com/fuchsia/+/master/third_party/rust_crates/ https://fuchsia.googlesource.com/fuchsia/+/master/third_part...
wocram 6y ago
What makes you think Google is using bazel for rust? I haven't seen or heard any evidence of this.
- gravypod 6y ago
  Internally at Google the vendor all source code and make it build using bazel. These rules also show some of the open source versions of this work: https://github.com/bazelbuild/rules_rust https://github.com/bazelbuild/rules_rust As well as the features that bazel provides you. The caching, packaging, code gen, cross-language deps/interfacing, and other magic that make bazel extremely amazing to use.

suprfnk 6y ago

Is this not also a function of being at least a /good/_ build system? If Cargo were too basic, or too hard to use, there would definitely be people creating a better one. This then causes the fragmentation you are talking about.

steveklabnik 6y ago
Yes, this matters too. I've been bumping into some tough limitations of Cargo lately, but it's still good enough to stick with it, and paper over them, rather than throw the whole thing out wholesale.
- mholub 6y ago
  what kind of limitations in a few words?
  - steveklabnik 6y ago
    https://users.rust-lang.org/t/complex-build-advice/48779 https://users.rust-lang.org/t/complex-build-advice/48779 is the best I can say at the moment.

pjmlp 6y ago

Not being able to deal with binary libraries and multiple languages is a big pain point for me, and no build.rs is not a solution, rather a workaround.

jnxx 6y ago
If it is a common package, you could use Guix to retrieve binary library packages for your architecture. (It will build from source if there is no pre-built package).
- pjmlp 6y ago
  Guix is a Linux only solution, and the whole point is not to build anything from 3rd party dependencies.
  - jnxx 6y ago
    > Guix is a Linux only solution, and the whole point is not to build anything from 3rd party dependencies. First, GNU Guix requires a POSIX system, and it will happily run e.g. on top of Debian in a suitable VM. (It will also run as a stand-alone OS which is nice if you want a bootstrapped, totally deterministic system - not good for gaming or casual development yet but there are serious long-term applications for that.) Also, as said it will build from source only if there is no pre-built binary package available for your platform, otherwise it will use cached binaries. That's, for example, a great use if you need to use cargo with a complex multi-architecture set-up. Just don't forgot to add plenty of disk space - here is a good use for Terabytes-size disks. Further, you can also set up your own build server, which caches binaries. The decision that it depends on the source code being available is rooted in the philosophy of the project; Guix is a GNU project and is intended to support Free software, as in FLOSS. I think this is also technically a good decision, because if the source code is not available, this leads sooner or later to practically unsolvable problems; think in all the expensive lab equipment in science which runs on decades old, obsolete, unsupported, unsafe platforms just because that expensive hardware requires a binary driver which won't run on more modern platforms. Or think in your scanner that stopped working after the last OS upgrade because the scanner vendor thought you ought buy a new one; that scanner will probably work fine under GNU/Linux, just as an illustration to the point being made. The need to make software reproducible is especially important in science where the scientific method depends on that experiments, and their evaluation can be repeated, but there is just no budget to port everything to the newest Python3.x, or OpenGL Y, or Qt-Z, or Matlab-ω. Science is competitive, no less than business, and you don't make a career in science with porting old stuff. But back to "Linux only" - right, there are platforms that Guix will not work on, like Windows. But complaining about that seems, at best, extremely naïve to me: By anyone who has the slightest knowledge in the matter it is well known that these proprietary platforms have literally spent decades on engineering to lock users in into their proprietary solutions. (And I think that the situation especially with C++ is in part a result of that, there is no inevitable technical reason that C++ does not have a standard ABI, or standard library discovery, it was just made that way to stop competitors similar to Borland's Delphi.) Complaining that their OS (while claiming POSIX compatibility) does not support GNU Guix is justs as, perdon my french, just as silly as complaining that you can't fix the broken battery of your MacBook yourself by taking a screwdriver and replacing it with a standard part. You won't have a screwdriver that fits precisely because the MacBook's vendor thought that you are not supposed to fix it yourself. Of course, the marketing department will tell you that their platform is "compatible to open source" - I have a bridge to sell you. These systems are just not designed for that, and the inability to use Guix on them is a consequence of your decision to use such a system; you are the one who will pay the costs of that, by having an obsolete scanner, or, far more expensive, by having obsolete, unmaintainable software years from now which is too expensive to port to a new supporting API.