7 ms·

Studies have shown that more than 87% of complaints about rust compile time are because people are enabling serde everywhere and for everything.

johnfn 5y ago
These concerns are valid. Data serialization/deserialization is an extremely standard thing to need to be able to do. It's not the user's fault if they use the obvious tool for an obvious job and suddenly have bloated compile times.
- vlovich123 5y ago
  Seems like there’s opportunity for tighter integration with the compiler where serde moves into the compiler or there are acceleration primitives rather than doing the AST macros thing. This sounds similar to the problems of c++ template bloat.
  - pjmlp 5y ago
    Except C++ template bloat has solutions not yet available in Rust. For example you can pre-instatiate the most common type parameters and just stuck those instantiations into a binary library. Also if you have a nice C++ compiler environment like Visual C++, you can even do edit-and-continue for many kinds of changes, and VS 2022 will double down on that capability.
    - vlovich123 5y ago
      C++ has theoretical solutions to it. I’ve yet to see anyone actually do anything like that in any professional environment (from small startup codebase a to the largest c++ code bases that exist in the world) because that’s an unmaintainable solution with a high burden placed on the developer. Whatever the solution Rust comes up with I think will look at the practical component so as to actually see adoption. It seems like the Rust team is very good at studying the mistakes other languages make in their approaches to a problem and finding the right fit within the language/targeted domain.
      - pjmlp 5y ago
        We used to do such stuff at Nokia, on HP-UX with Clearcase and .o build caching supported by cleartool using aCC, while working on NetAct, professional enough?
        vlovich123 5y ago
        My point was more meant to be that it's extremely rare, not that it doesn't ever happen. Do people partially specialize templates? I'm sure yes, it happens from time to time. Lots of things happen from time-to-time depending on the mix of the team & the problem domain. If you picked a random team/codebase, how many do you think are carefully optimizing partial template specialization? Moreover, what tools even exist to let you profile the contribution of a missing specialization and measure the impact of your change? Not sure how .o build caching relates to partial template specialization though. With regards to your specific experience, I don't know that I'd say that Clearcase, HP-UX or aCC are main-stream professional C++ development environments. I think for that you're really looking at clang, gcc, & msvc with icc being a distant fourth that has itself moved to clang recently. The primary target OS would either be server Linux, Android, macOS, iOS with maybe FreeBSD picking up the slack. HP-UX has a niche but it's a small niche comparatively in terms of $ spent on developers in that ecosystem. In terms of source control, everyone is mostly on git or mercurial. Anyone not just refuses to get with the times because they have bought into their existing tool expertise but don't know how to adapt.
kkwteh 5y ago
That's great. That means that if the Rust compiler is made faster in this one case it'll make a huge difference across the board.
brabel 5y ago
@burntsushi had a post where he explained that the common error-handling crates also add big compile-time costs. Most projects use them as well as serde, so the problem compounds.
- mwcampbell 5y ago
  Can you provide a link to that post?
  - brabel 5y ago
    https://www.reddit.com/r/rust/comments/gj8inf/rust_structuring_and_handling_errors_in_2020/fqlmknt/ https://www.reddit.com/r/rust/comments/gj8inf/rust_structuri...
- ahahahahah 5y ago
  I think that's a misunderstanding. That's a very different problem from the serde one. The compile time costs for the error handling crates is due to increase in build graph size, not due to the cost of running the error handling proc macros or the cost of compiling the code generated from them. For anything but a small project, that increase in graph size is probably not a big deal (and likely the project is already using much of the same deps). This cost does not increase as a project gets larger. serde also probably has the build graph size problem, but the bigger problem is the cost generating and compiling the generated code. This cost tends to increase as a project gets larger.

johnfn 5y ago

These concerns are valid. Data serialization/deserialization is an extremely standard thing to need to be able to do. It's not the user's fault if they use the obvious tool for an obvious job and suddenly have bloated compile times.

vlovich123 5y ago
Seems like there’s opportunity for tighter integration with the compiler where serde moves into the compiler or there are acceleration primitives rather than doing the AST macros thing. This sounds similar to the problems of c++ template bloat.
- pjmlp 5y ago
  Except C++ template bloat has solutions not yet available in Rust. For example you can pre-instatiate the most common type parameters and just stuck those instantiations into a binary library. Also if you have a nice C++ compiler environment like Visual C++, you can even do edit-and-continue for many kinds of changes, and VS 2022 will double down on that capability.
  - vlovich123 5y ago
    C++ has theoretical solutions to it. I’ve yet to see anyone actually do anything like that in any professional environment (from small startup codebase a to the largest c++ code bases that exist in the world) because that’s an unmaintainable solution with a high burden placed on the developer. Whatever the solution Rust comes up with I think will look at the practical component so as to actually see adoption. It seems like the Rust team is very good at studying the mistakes other languages make in their approaches to a problem and finding the right fit within the language/targeted domain.
    - pjmlp 5y ago
      We used to do such stuff at Nokia, on HP-UX with Clearcase and .o build caching supported by cleartool using aCC, while working on NetAct, professional enough?
      - vlovich123 5y ago
        My point was more meant to be that it's extremely rare, not that it doesn't ever happen. Do people partially specialize templates? I'm sure yes, it happens from time to time. Lots of things happen from time-to-time depending on the mix of the team & the problem domain. If you picked a random team/codebase, how many do you think are carefully optimizing partial template specialization? Moreover, what tools even exist to let you profile the contribution of a missing specialization and measure the impact of your change? Not sure how .o build caching relates to partial template specialization though. With regards to your specific experience, I don't know that I'd say that Clearcase, HP-UX or aCC are main-stream professional C++ development environments. I think for that you're really looking at clang, gcc, & msvc with icc being a distant fourth that has itself moved to clang recently. The primary target OS would either be server Linux, Android, macOS, iOS with maybe FreeBSD picking up the slack. HP-UX has a niche but it's a small niche comparatively in terms of $ spent on developers in that ecosystem. In terms of source control, everyone is mostly on git or mercurial. Anyone not just refuses to get with the times because they have bought into their existing tool expertise but don't know how to adapt.

kkwteh 5y ago

That's great. That means that if the Rust compiler is made faster in this one case it'll make a huge difference across the board.

brabel 5y ago

@burntsushi had a post where he explained that the common error-handling crates also add big compile-time costs. Most projects use them as well as serde, so the problem compounds.

mwcampbell 5y ago
Can you provide a link to that post?
- brabel 5y ago
  https://www.reddit.com/r/rust/comments/gj8inf/rust_structuring_and_handling_errors_in_2020/fqlmknt/ https://www.reddit.com/r/rust/comments/gj8inf/rust_structuri...
ahahahahah 5y ago
I think that's a misunderstanding. That's a very different problem from the serde one. The compile time costs for the error handling crates is due to increase in build graph size, not due to the cost of running the error handling proc macros or the cost of compiling the code generated from them. For anything but a small project, that increase in graph size is probably not a big deal (and likely the project is already using much of the same deps). This cost does not increase as a project gets larger. serde also probably has the build graph size problem, but the bigger problem is the cost generating and compiling the generated code. This cost tends to increase as a project gets larger.