3 ms·

Because it demonstrates a failure of the Rust ecosystem. Proc macros have effectively become fundamental to Rust. This is unfortunate, but, given that it is r

by bsder 3y ago

Because it demonstrates a failure of the Rust ecosystem. Proc macros have effectively become fundamental to Rust. This is unfortunate, but, given that it is reality, they should get compiler support. Part of what is making them so slow is that they need to process a firehose of information through a drinking straw. Placing proc macros into the compiler would not only speed them up but also remove the 50+ crate dependency chain that gets pulled in. Serde, while slow, has a more fundamental Rust problem. Serde demonstrates that Rust needs some more abstractions that it currently does not have. The "orphan rule" means that you, dear user, cannot take my crate and Serde and compose them without a large amount of copypasta boilerplate. A better composition story would speed up the compilation of Serde as it would have to do far less work. Rust has fundamental problems to solve, and lately only seems capable of stopgaps instead of progress. This is unfortunate as it will wind up ceding the field to something like Carbon.

veber-alex 3y ago
proc macros are user written code, how can you "place them into the compiler"? What does the orphan rule have to do with the compilation speed of serde? How does it reduce the work it needs to do?
- bsder 3y ago
  You have to pulverize the macro into syntax atoms and then reassemble it piece by piece into an AST that the compiler needs. All the processing to do all of that should be a part of the compiler--not a smattering of 50 crates (syn being the most oppressive one) and a weak compiler API. Serde has to enumerate the universe even for types that wind up not used. Those types then need to be discarded--which is problematic if it takes a non-trivial amount of time to enumerate or construct them (say: by using proc macros). Part of this is the specification that must be done because Rust can't do compile-time reflection/introspection part of which is blocked because of the way Rust resolves types--different versions of crates wind up with different types even if the structures are structurally identical because Rust can't do compile time reflection and relies on things like the Orphan rule. I am not saying that any of these things are easy. They are in fact hard. However, without solving them, Rust compilation times will always be garbage. I'm beginning to think that the folks who claimed that compiler speed is the single most important design criterion were right. It seems like you can't go back and retrofit "faster compiler time" afterward.
  - veber-alex 3y ago
    > You have to pulverize the macro into syntax atoms and then reassemble it piece by piece into an AST that the compiler needs. All the processing to do all of that should be a part of the compiler--not a smattering of 50 crates (syn being the most oppressive one) and a weak compiler API. First of all it's not 50 crates, it's 3 main crates syn, quote and proc-macro2 and a few smaller helper crates which are used less often. And the reason to not include them in the compiler is exactly the same reason why "crate X" is not included in std. Stuff in std must be maintained forever without breaking changes. Just recently syn was bumped to version 2 with breaking changes which would have not been possible if it was part of the compiler. In any case, shipping precompiled proc macros WASM binaries will solve this problem. > Serde has to enumerate the universe even for types that wind up not used. Those types then need to be discarded--which is problematic if it takes a non-trivial amount of time to enumerate or construct them (say: by using proc macros). Part of this is the specification that must be done because Rust can't do compile-time reflection/introspection... Yes, it's a design tradeoff, in return Rust has no post-monomorphization errors outside of compile time function evaluation. I think it's worth it. > I'm beginning to think that the folks who claimed that compiler speed is the single most important design criterion were right. I don't agree. As long as the compile times are reasonable (and I consider Rust compile times to be more than reasonable) other aspects of the language are more important. Compile time is a tradeoff and should be balanced against other features carefully, it's not the end goal of a language.
    - bsder 3y ago
      > First of all it's not 50 crates, it's 3 main crates syn, quote and proc-macro2 and a few smaller helper crates which are used less often. That may be technically true. However, every time I pull in something that uses proc macros it seems my dependency count goes flying through the roof and my compile times go right out the door. Maybe people aren't using proc macros properly. That's certainly possible. But, that also speaks to the ecosystem, as well.
      - Ygg2 3y ago
        > seems my dependency count goes flying through the roof and my compile times go right out the door. I have two dependencies (memchr, urlencoding) and one two dev dependencies (libmimic, criterion). I have 77 transitive dependencies, go figure. People like to outsource stuff to other libraries, when possible.
kibwen 3y ago
> Rust has fundamental problems to solve The orphan rule isn't a problem that needs solving, it is itself a solution to a problem. There's nothing stopping Rust from removing the orphan rule right this minute and introducing bold new problems regarding trait implementation coherence.
- bsder 3y ago
  > The orphan rule isn't a problem that needs solving, it is itself a solution to a problem. It is both. The Orphan Rule was indeed chosen as a solution because other choices were dramatically more difficult to implement. However, the Orphan Rule also has tradeoffs that make certain things problematic. Engineering is tradeoffs. There are only two types of programming languages: those you bitch about and those you don't use. YMMV. etc.
  - Twey 3y ago
    Edward Kmett has a great explanation[1] of the considerations around this space. It's not that alternatives to the orphan rule are hard to implement, it's that they lead to things being much harder to reason about for the programmer. [1]: https://youtube.com/watch?v=hIZxTQP1ifo https://youtube.com/watch?v=hIZxTQP1ifo
dundarious 3y ago
I strongly agree with all your criticisms and factual assertions, but I think all your predictions are likely to end up being wrong.
- bsder 3y ago
  I hope you are correct about predictions because I really want to see the day that C++ gets a spike put through it because something better exists. I really don't want to see something like Carbon take off. It's not really that much "better" than C++, and it will take up the oxygen from something that could be a genuine replacement.

by bsder 3y ago

veber-alex 3y ago

proc macros are user written code, how can you "place them into the compiler"? What does the orphan rule have to do with the compilation speed of serde? How does it reduce the work it needs to do?

bsder 3y ago
You have to pulverize the macro into syntax atoms and then reassemble it piece by piece into an AST that the compiler needs. All the processing to do all of that should be a part of the compiler--not a smattering of 50 crates (syn being the most oppressive one) and a weak compiler API. Serde has to enumerate the universe even for types that wind up not used. Those types then need to be discarded--which is problematic if it takes a non-trivial amount of time to enumerate or construct them (say: by using proc macros). Part of this is the specification that must be done because Rust can't do compile-time reflection/introspection part of which is blocked because of the way Rust resolves types--different versions of crates wind up with different types even if the structures are structurally identical because Rust can't do compile time reflection and relies on things like the Orphan rule. I am not saying that any of these things are easy. They are in fact hard. However, without solving them, Rust compilation times will always be garbage. I'm beginning to think that the folks who claimed that compiler speed is the single most important design criterion were right. It seems like you can't go back and retrofit "faster compiler time" afterward.
- veber-alex 3y ago
  > You have to pulverize the macro into syntax atoms and then reassemble it piece by piece into an AST that the compiler needs. All the processing to do all of that should be a part of the compiler--not a smattering of 50 crates (syn being the most oppressive one) and a weak compiler API. First of all it's not 50 crates, it's 3 main crates syn, quote and proc-macro2 and a few smaller helper crates which are used less often. And the reason to not include them in the compiler is exactly the same reason why "crate X" is not included in std. Stuff in std must be maintained forever without breaking changes. Just recently syn was bumped to version 2 with breaking changes which would have not been possible if it was part of the compiler. In any case, shipping precompiled proc macros WASM binaries will solve this problem. > Serde has to enumerate the universe even for types that wind up not used. Those types then need to be discarded--which is problematic if it takes a non-trivial amount of time to enumerate or construct them (say: by using proc macros). Part of this is the specification that must be done because Rust can't do compile-time reflection/introspection... Yes, it's a design tradeoff, in return Rust has no post-monomorphization errors outside of compile time function evaluation. I think it's worth it. > I'm beginning to think that the folks who claimed that compiler speed is the single most important design criterion were right. I don't agree. As long as the compile times are reasonable (and I consider Rust compile times to be more than reasonable) other aspects of the language are more important. Compile time is a tradeoff and should be balanced against other features carefully, it's not the end goal of a language.
  - bsder 3y ago
    > First of all it's not 50 crates, it's 3 main crates syn, quote and proc-macro2 and a few smaller helper crates which are used less often. That may be technically true. However, every time I pull in something that uses proc macros it seems my dependency count goes flying through the roof and my compile times go right out the door. Maybe people aren't using proc macros properly. That's certainly possible. But, that also speaks to the ecosystem, as well.
    - Ygg2 3y ago
      > seems my dependency count goes flying through the roof and my compile times go right out the door. I have two dependencies (memchr, urlencoding) and one two dev dependencies (libmimic, criterion). I have 77 transitive dependencies, go figure. People like to outsource stuff to other libraries, when possible.

kibwen 3y ago

> Rust has fundamental problems to solve The orphan rule isn't a problem that needs solving, it is itself a solution to a problem. There's nothing stopping Rust from removing the orphan rule right this minute and introducing bold new problems regarding trait implementation coherence.

bsder 3y ago
> The orphan rule isn't a problem that needs solving, it is itself a solution to a problem. It is both. The Orphan Rule was indeed chosen as a solution because other choices were dramatically more difficult to implement. However, the Orphan Rule also has tradeoffs that make certain things problematic. Engineering is tradeoffs. There are only two types of programming languages: those you bitch about and those you don't use. YMMV. etc.
- Twey 3y ago
  Edward Kmett has a great explanation[1] of the considerations around this space. It's not that alternatives to the orphan rule are hard to implement, it's that they lead to things being much harder to reason about for the programmer. [1]: https://youtube.com/watch?v=hIZxTQP1ifo https://youtube.com/watch?v=hIZxTQP1ifo

dundarious 3y ago

I strongly agree with all your criticisms and factual assertions, but I think all your predictions are likely to end up being wrong.

bsder 3y ago
I hope you are correct about predictions because I really want to see the day that C++ gets a spike put through it because something better exists. I really don't want to see something like Carbon take off. It's not really that much "better" than C++, and it will take up the oxygen from something that could be a genuine replacement.