4 ms·

> have easy paralelisation of builds. except the company had a straight forward solution to the problem and it's not always possible in C++ land either and C

> have easy paralelisation of builds. except the company had a straight forward solution to the problem and it's not always possible in C++ land either and COM isn't part of C/C++ but a microsoft specific extension which solves very different issues as it's for cross application communication while we here have compile time perf issues inside of a single library > binary libraries I'm not sure if you mean dynamic linking or binary objects but there is the thing: - dynamic linking isn't the issue here as the issue has nothing to do with re-compilation and similar (where dynamic linking can help), - binary object files on the other hand are also something rust has and uses, it just sets the boundaries in different places (crate instead of file) which makes development easier, can lead to better runtime performance etc. It just has the drawback that you sometimes have to split things into multiple crates. > a language as complex as how complex a language is to write has not too much to do with how complex it is to split a single code unit into multiple parts for parallel compilation. C, C++ and D mainly side steps this by making each file a compilation unit, while in rust it's each crate. But that isn't fundamentally better or worse, it's trade offs. It's trade offs. > because Walter Bright did the right decisions on what to focus for development workflow. and so did rust, just with different priorities and trade offs and given that D is mostly irrelevant and rust increasingly more successful maybe it was pursing the more important priorities Also the OP case is about release builds i.e. not the normal dev loop neither does the splitting affect dev experience as it's all auto generated code and if projects which aren't auto generated it's very normal to split out libraries etc. anyway, and weather you split them into their own module, file or crate doesn't matter too much as long as you keep code clean (as in not having supper entangled Spagetti code) and I wouldn't even be sure if D does perform in any relevant way better in compiler time if you compare their performance with the end result after they split the crate

pjmlp 2y ago
COM is one approach, among others like SOM, DCE, ... as means to write binary libraries in poliglot languages. Turns out that it is mostly used on Windows since Vista days, as means to have an OOP based OS, with most libraries written in C++, and having a stable ABI for such components. So while it isn't ISO C++, it is mostly used by and from C++. .NET land usually only reaches for COM when using Windows APIs. Binary libraries, mean binary libraries, doesn't matter if static or dynamically linked. My point is that Rust ecosystem does not use them, you always need to compile from source code the complete dependency tree after a git clone, some of them even multiple times due to different feature flags configurations. Not so with most commercial C and C++ development, we enjoy having binary libraries for dependecies, after a git clone only the main code needs to be compiled from scratch. Yes there are ways to kind of do with sccache, but it is additionally tooling, not something that apparently cargo will ever support. Also if you watch recent talks from Microsoft regarding their Rust adoption, the lack of tooling support for binary libraries distribution is one of their pain points.
- dathinab 2y ago
  you are mixing up use cases and concepts mainly the concepts of - how build steps are cached in on instance of the project - how build steps are cached across projects of common dependencies - linking to system libraries - bundling dependencies -- Lets first look at it from a POV of system dependencies vs. bundling dependencies: For each dependency you either link them, making them a system dependencies (because you link against the on in you system and require systems to have them) or bundle them, doesn't matter if it's C or Rust. The problem with system dependencies is they don't just need API compatibility but also ABI compatibility and not everything with ABI compatibility is actually API compatible. Which is all nice and fun except a ton of thing highly useful (some would say required) do not work well (or at all) if you need ABI compatibility and there are tone of (potential security bugs) seemingly API compatible but not API compatible libraries has. In general history has shown that making most dependencies system dependencies is a complete shit show not worth anyone time and money, especially if people start mixing versions which seem compatible but aren't leading to strange runtime bug aren't possible with supported builds but anyway somehow are your fault as library maintainer. Which is why the huge majority of the software industry _gave up on them_ for anything where they aren't strictly needed. Rust can produce and use system dependencies, using a C API and with some less officially supported way also using rlibs (i.e. the binary libraries rust produces when compiling a crate, so yes it's using binary libraries). But mostly it's not worth bothering with it, in the same way the majority of the rest of the software ecosystem stopped doing it. -- Then let's look at reusing builds i.e. caching. By default rust does that, but only on the scope of the project. I.e. you build a project change something and then rebuild it and dependencies won't be build again (except if you need to rebuild them I come back to it later). To be clear this is _not_ incremental building, which is a feature to re-use build parts on a more granular level then crates. If you want it to cache things across projects or with some company build server you can do so using 3rd party software, i.e. same situation as with C. > most commercial C and C++ development Committing binary build artifact to a source code repo is a huge anti pattern, a terrible way to have distributed build caches. Stuff like that can easily make your company fail security reviews or become classified as having acted negligently if sued for damaged (e.g. caused by a virus sneaked into your program). Also please _never ever_ checkout a 3rd party open source project with any pre-build binary artifacts in it, it's a huge security threat. So in C/C++ you also should use the additional tools. > Rust ecosystem does not use them as mentioned they produce rlibs which are binary libraries (or you could say binary libraries bundled with metadata, and stuff which is roughly like how C++ templates are handled wrt. binary libraries) And yes the tooling for shipping pre-build rlibs could be better, and it probably will get better. It's not that it can't be done, just priorities have been elsewhere so far. > even multiple times due to different feature flags configurations. Features are strictly additive, so no that won't happen. The only reason for them being build multiple times is different incompatible versions of the package (which from rust POV are two different dependencies altogether). And while that seems initially kinda dump (unnecessary binary size/build time) but I can't understate how much of a huge blessing this turned out to be. > not something that apparently cargo will ever support. yes and make doesn't support distributed build caches without including 3rd party tools either. But it doesn't matter as long as you can just pull in the 3rd party tools if you need them. EDIT: Rust features is like using #if and similar in the C/C++ pre-processor, i.e. if they change you have to rebuild. Like in C/C++. Also even without a crate might have been only partially compiled before (e.g. only 1 of 3 functions) so if you start using the other parts they still need to be compiled which will look a lot like recompilation (and without the incremental build feature might be a rebuild).

pjmlp 2y ago

COM is one approach, among others like SOM, DCE, ... as means to write binary libraries in poliglot languages. Turns out that it is mostly used on Windows since Vista days, as means to have an OOP based OS, with most libraries written in C++, and having a stable ABI for such components. So while it isn't ISO C++, it is mostly used by and from C++. .NET land usually only reaches for COM when using Windows APIs. Binary libraries, mean binary libraries, doesn't matter if static or dynamically linked. My point is that Rust ecosystem does not use them, you always need to compile from source code the complete dependency tree after a git clone, some of them even multiple times due to different feature flags configurations. Not so with most commercial C and C++ development, we enjoy having binary libraries for dependecies, after a git clone only the main code needs to be compiled from scratch. Yes there are ways to kind of do with sccache, but it is additionally tooling, not something that apparently cargo will ever support. Also if you watch recent talks from Microsoft regarding their Rust adoption, the lack of tooling support for binary libraries distribution is one of their pain points.

dathinab 2y ago
you are mixing up use cases and concepts mainly the concepts of - how build steps are cached in on instance of the project - how build steps are cached across projects of common dependencies - linking to system libraries - bundling dependencies -- Lets first look at it from a POV of system dependencies vs. bundling dependencies: For each dependency you either link them, making them a system dependencies (because you link against the on in you system and require systems to have them) or bundle them, doesn't matter if it's C or Rust. The problem with system dependencies is they don't just need API compatibility but also ABI compatibility and not everything with ABI compatibility is actually API compatible. Which is all nice and fun except a ton of thing highly useful (some would say required) do not work well (or at all) if you need ABI compatibility and there are tone of (potential security bugs) seemingly API compatible but not API compatible libraries has. In general history has shown that making most dependencies system dependencies is a complete shit show not worth anyone time and money, especially if people start mixing versions which seem compatible but aren't leading to strange runtime bug aren't possible with supported builds but anyway somehow are your fault as library maintainer. Which is why the huge majority of the software industry _gave up on them_ for anything where they aren't strictly needed. Rust can produce and use system dependencies, using a C API and with some less officially supported way also using rlibs (i.e. the binary libraries rust produces when compiling a crate, so yes it's using binary libraries). But mostly it's not worth bothering with it, in the same way the majority of the rest of the software ecosystem stopped doing it. -- Then let's look at reusing builds i.e. caching. By default rust does that, but only on the scope of the project. I.e. you build a project change something and then rebuild it and dependencies won't be build again (except if you need to rebuild them I come back to it later). To be clear this is _not_ incremental building, which is a feature to re-use build parts on a more granular level then crates. If you want it to cache things across projects or with some company build server you can do so using 3rd party software, i.e. same situation as with C. > most commercial C and C++ development Committing binary build artifact to a source code repo is a huge anti pattern, a terrible way to have distributed build caches. Stuff like that can easily make your company fail security reviews or become classified as having acted negligently if sued for damaged (e.g. caused by a virus sneaked into your program). Also please _never ever_ checkout a 3rd party open source project with any pre-build binary artifacts in it, it's a huge security threat. So in C/C++ you also should use the additional tools. > Rust ecosystem does not use them as mentioned they produce rlibs which are binary libraries (or you could say binary libraries bundled with metadata, and stuff which is roughly like how C++ templates are handled wrt. binary libraries) And yes the tooling for shipping pre-build rlibs could be better, and it probably will get better. It's not that it can't be done, just priorities have been elsewhere so far. > even multiple times due to different feature flags configurations. Features are strictly additive, so no that won't happen. The only reason for them being build multiple times is different incompatible versions of the package (which from rust POV are two different dependencies altogether). And while that seems initially kinda dump (unnecessary binary size/build time) but I can't understate how much of a huge blessing this turned out to be. > not something that apparently cargo will ever support. yes and make doesn't support distributed build caches without including 3rd party tools either. But it doesn't matter as long as you can just pull in the 3rd party tools if you need them. EDIT: Rust features is like using #if and similar in the C/C++ pre-processor, i.e. if they change you have to rebuild. Like in C/C++. Also even without a crate might have been only partially compiled before (e.g. only 1 of 3 functions) so if you start using the other parts they still need to be compiled which will look a lot like recompilation (and without the incremental build feature might be a rebuild).