3 ms·

I agree that pydantic could have been faster while still being written in Python. The argument for Rust: 1. If I'm going to rewrite - why not go the whole hog

I agree that pydantic could have been faster while still being written in Python. The argument for Rust: 1. If I'm going to rewrite - why not go the whole hog and do it in rust - and thereby get a 20x improvement, not 2x. 2. By using Rust we can have add more customisation with virtually no performance impact, with Python that's not the case. Of course we could make Pydantic faster by removing features, but that would be very disappointing for existing user. As mentioned by other commenters, your comment about "constrained" does not apply.

andreareina 3y ago
> If I'm going to rewrite - why not go the whole hog and do it in rust We use black at work. One of the challenges with it is that it doesn't play very nicely with pandas, specifically its abundant use of []. So we forked it and maintain a trivial patch that treats '[' the same as '.' and everybody's happy. What was maybe 15 minutes of work for me to get everybody's buy-in to use a formatter would not have been so quick or easy if it had been written in rust and now either we maintain our own repo of binary wheels, or all our devs now need to include rust in their build tooling. I'm not invested in the argument one way or the other, just wanted to note that having the stack be accessible to easy modification by any user is itself a feature and one some people (including me in general, not so much in this particular case) derive a lot of benefit from. P.S. cheers and congratulations!
- heyoni 3y ago
  This is so on point but is already the case with any package written in C. I feel like there’s such a strong push towards having rust backends for python packages that you might have to learn it to become a decent python developer…and I think I might be ok with that. For the price of having 1 dev on your team understand rust, you can keep using python as a top performing language. We’ve got Ruff, Pydantic and Rye (experimental?) just off the top of my head being written in rust. It seems like that’s where we’re heading as a community.
  - andreareina 3y ago
    I'm talking about the rest of my team being able to use the fork. A compiler and toolchain is much more likely to be installed for c than for rust.
    - carlmr 3y ago
      With Rust drivers in the Linux kernel I don't think this will be for long.
      - andreareina 3y ago
        Good point, and gcc is getting a roast front end as well!
dnsco 3y ago
I really appreciate your transparency around, "I am the one who am writing this open source library, and I think it will be more fun to do it this way." Have fun! I truly hope it pays the returns you hope it will as well. Naysayers: you're welcome to fork the old python version. If the rust version is a nightmare for the ecosystem, I'm sure someone will do that. I'm very excited to see the results!
antman 3y ago
Because now you are on your own looking at the community from afar. You have taken the Drupal path and the ones who can help you are also not being helped in their rust paved paths so they are busy. Strange how it turned out that way, at the last convention everyone agreed that the best tool for python is rust... The silent majority was not there. At least do it in Nim, a python dev can quickly catch up. Optimization kills resilience.

andreareina 3y ago

> If I'm going to rewrite - why not go the whole hog and do it in rust We use black at work. One of the challenges with it is that it doesn't play very nicely with pandas, specifically its abundant use of []. So we forked it and maintain a trivial patch that treats '[' the same as '.' and everybody's happy. What was maybe 15 minutes of work for me to get everybody's buy-in to use a formatter would not have been so quick or easy if it had been written in rust and now either we maintain our own repo of binary wheels, or all our devs now need to include rust in their build tooling. I'm not invested in the argument one way or the other, just wanted to note that having the stack be accessible to easy modification by any user is itself a feature and one some people (including me in general, not so much in this particular case) derive a lot of benefit from. P.S. cheers and congratulations!

heyoni 3y ago
This is so on point but is already the case with any package written in C. I feel like there’s such a strong push towards having rust backends for python packages that you might have to learn it to become a decent python developer…and I think I might be ok with that. For the price of having 1 dev on your team understand rust, you can keep using python as a top performing language. We’ve got Ruff, Pydantic and Rye (experimental?) just off the top of my head being written in rust. It seems like that’s where we’re heading as a community.
- andreareina 3y ago
  I'm talking about the rest of my team being able to use the fork. A compiler and toolchain is much more likely to be installed for c than for rust.
  - carlmr 3y ago
    With Rust drivers in the Linux kernel I don't think this will be for long.
    - andreareina 3y ago
      Good point, and gcc is getting a roast front end as well!

dnsco 3y ago

I really appreciate your transparency around, "I am the one who am writing this open source library, and I think it will be more fun to do it this way." Have fun! I truly hope it pays the returns you hope it will as well. Naysayers: you're welcome to fork the old python version. If the rust version is a nightmare for the ecosystem, I'm sure someone will do that. I'm very excited to see the results!

antman 3y ago

Because now you are on your own looking at the community from afar. You have taken the Drupal path and the ones who can help you are also not being helped in their rust paved paths so they are busy. Strange how it turned out that way, at the last convention everyone agreed that the best tool for python is rust... The silent majority was not there. At least do it in Nim, a python dev can quickly catch up. Optimization kills resilience.