6 ms·

Did not take long at all to see Pydantic version 2 support. Nice!

nerdponx 3y ago
I still can't stand Pydantic's API and its approach to non-documentation. I respect the tremendous amount of hard work that goes into it, but fundamentally I don't like the developer experience and I don't think I'll ever feel otherwise. I use it because my coworkers like it and I've learned its advanced features because I had to in order to get things done, not because I like it. I would love to see a FastAPI alternative still using Starlette internally, but using Attrs + Marshmallow + Cattrs + Apispec instead of Pydantic. It would be a little less "fast" to write a working prototype, but I'd feel much more comfortable working with those APIs, as well as much more comfortable that my dependencies are well-supported and stable. The problem of course is not that gluing those things together is hard. The problem is that now someone has put untold hundreds of person-hours into FastAPI, and replicating that level of care, polish, bugfixes, feature requests, etc. is difficult without putting in those hundreds of person-hours yourself.
- willhoyle 3y ago
  I too have made similar observations regarding pydantic and FastAPI. I was evaluating various Python async http frameworks and landed on a similar stack: - attrs/cattrs for models - starlette+uvicorn for HTTP/websocket - validation I’m still on the fence about. I’ll see how far I get with the built in validators offered by attrs. I use voluptuous at work and generally like the DX but it’s in maintenance mode. This is purely personally preference, I’m sure devs using fastapi+pydantic are more productive in the long run. It almost feels like I’m hand rolling my own fastapi implementation but at the same time I don’t want to be too locked in to frameworks like that. Ive been burnt by magic frameworks that do too much behind the scenes and there’s something nice about fully understanding what’s going on when you hand stitch libraries yourself.
  - jammycrisp 3y ago
    If you like cattrs, you _might_ be interested in trying out my msgspec library [1]. It works out-of-the-box with attrs objects (as well as its own faster `Struct` types), while being ~10-15x faster than cattrs for encoding/decoding/validating JSON. The hope is it's easy to integrate msgspec with other tools (like attrs!) rather than forcing the user to rewrite code to fit the new validation/serialization framework. It may not fit every use case, but if msgspec works for you it should be generally an order-of-magnitude faster than other Python options. [1]: https://github.com/jcrist/msgspec https://github.com/jcrist/msgspec </blatant-evangelism>
    - willhoyle 3y ago
      This looks like exactly what I've been looking for. I just want strong typing, json <-> struct and validation. Seems like it ticks all the boxes + speed benefits which is always nice. I especially find it useful that I can use messagepack for internal service chatter but still support json for external stuff and dump astuple to sqlite.
  - eyegor 3y ago
    Depending on how far in you are, starlite/litestar has good documentation and offers another "batteries included" framework. Performance wise it's about the same and the stack is about the same. Fastapi suffers from the "one solo dev in Nebraska" paradigm (check out open prs and old tickets). For me the main draw of litestar is the batteries + better docs + more active development with multiple developers vs most other python web frameworks.
    - jammycrisp 3y ago
      +1 for litestar[1]. The higher bus-factor is nice, and I like that they're working to embrace a wider set of technologies than just pydantic. The framework currently lets you model objects using msgspec[2] (they actually use msgspec for all serialization), pydantic, or attrs[3], and the upcoming release adds some new mechanisms for handling additional types. I really appreciate the flexibility in modeling APIs; not everything fits well into a pydantic shaped box. [1]: https://litestar.dev/ https://litestar.dev/ [2]: https://github.com/jcrist/msgspec https://github.com/jcrist/msgspec [3]: https://www.attrs.org/en/stable/ https://www.attrs.org/en/stable/
    - nerdponx 3y ago
      I haven't heard of Starlite or Litestar before. Is one a fork of the other? Their documentation intro text is identical: > {Litestar|Starlite} is a powerful, flexible, highly performant, and opinionated ASGI framework, offering first class typing support and a full Pydantic integration. > > The {Litestar|Starlite} framework supports Plugins, ships with dependency injection, security primitives, OpenAPI schema generation, MessagePack, middlewares, and much more.
      - jammycrisp 3y ago
        starlite was the original name, it was recently renamed to litestar due to comments about how easily confused "starlette" and "starlite" are.
        d0mine 3y ago
        data point in favor of the renaming: while reading GP, I've assumed that "starlite" is a typo for "starlette."
- ramraj07 3y ago
  Could you simplify your point? I was an ardent marshmallow user and when I finally switched to pydantic, it felt like I finally sat down in my life after standing forever. The documentation sounds good enough to me, but importantly the interface pydantic provides to define your json schema is the most elegant interface I’ve seen in any language and miles better than the mess marshmallow provided. For many of us especially in the SaaS side, speed of these operations is a distant third priority compared to ease of writing and understanding the code, and ensuring reliable less buggy code. The actual compute happens on a cluster with spark or snowflake anyway.
  - nerdponx 3y ago
    There is no reference doc. The docs cover a lot of material in a small amount of space, buying important pieces of information and mixing up a large number of topics under unintuitive headlines. Reading the source code is occasionally necessary just to figure out how it all works. The API is a little weird, particularly around defining validators. The parameter name-matching is an "interesting" design choice. Accessing "values" as a dict[str,Any] is messy if you care about static typing, although I can understand why they did it. Furthermore, the behavior of validators and the exact sequence in which they run is not defined by the docs. It's not that hard to figure out, but it also might change at any time because there's no user contract. Attrs is significantly nicer in just about all respects here, especially their attention to detail in their extensive user guide and reference docs. Speaking of user contract, there's no clear separation between private and public. Without a reference doc it all looks like fair game, but without a reference doc it also might all change at any moment. Either you stick to the examples, or you're off doing a guess-and-check dance and hoping something doesn't break. Even with the Mypy plugin, I often have to write `if TYPE_CHECKING` all over any nontrivial Pydantic class consuming data from external sources. Variable annotations in Pydantic are fundamentally not PEP 484 type hints. That's fine, but it's confusing that they're almost the same, and, as above, it's almost entirely up to you to figure out how it all works, either by trial and error or by digging around in the issue tracker and StackOverflow. Ease of writing and reliability is precisely my big area of annoyance and concern. Speed of (de)serialization is comparatively unimportant (although I don't like the huge amount of overhead involved and I avoid using it in hot code paths). I also don't like using Pydantic-defined classes very much, because the actual init method signature is just *args, **kwargs, which doesn't work well with any tooling. It feels like being back in the Tornado & PyMongo dark ages where everything is dynamic or dynamically-generated and classes are just glorified hash tables. I agree that the JSONSchema integration is outstanding. BaseSettings is also a tremendous productivity improvement, I love that I can define a class and immediately get a proper app-wide config reading from both env vars and a dotenv file. I also like the default error messages that tell you exactly which field failed validation. I also like the validator system (once I figured out how it worked), respecting the order in which I define the validators as well as supporting validators that run before or after the default set of validators (pre=True and pre=False respectively). I was probably being a little too negative before, but my annoyance level with the developer-facing API and documentation remains high, and I will gladly jump to an Attrs-based alternative as soon as one exists.*

I still can't stand Pydantic's API and its approach to non-documentation. I respect the tremendous amount of hard work that goes into it, but fundamentally I don't like the developer experience and I don't think I'll ever feel otherwise. I use it because my coworkers like it and I've learned its advanced features because I had to in order to get things done, not because I like it. I would love to see a FastAPI alternative still using Starlette internally, but using Attrs + Marshmallow + Cattrs + Apispec instead of Pydantic. It would be a little less "fast" to write a working prototype, but I'd feel much more comfortable working with those APIs, as well as much more comfortable that my dependencies are well-supported and stable. The problem of course is not that gluing those things together is hard. The problem is that now someone has put untold hundreds of person-hours into FastAPI, and replicating that level of care, polish, bugfixes, feature requests, etc. is difficult without putting in those hundreds of person-hours yourself.

willhoyle 3y ago
I too have made similar observations regarding pydantic and FastAPI. I was evaluating various Python async http frameworks and landed on a similar stack: - attrs/cattrs for models - starlette+uvicorn for HTTP/websocket - validation I’m still on the fence about. I’ll see how far I get with the built in validators offered by attrs. I use voluptuous at work and generally like the DX but it’s in maintenance mode. This is purely personally preference, I’m sure devs using fastapi+pydantic are more productive in the long run. It almost feels like I’m hand rolling my own fastapi implementation but at the same time I don’t want to be too locked in to frameworks like that. Ive been burnt by magic frameworks that do too much behind the scenes and there’s something nice about fully understanding what’s going on when you hand stitch libraries yourself.
- jammycrisp 3y ago
  If you like cattrs, you _might_ be interested in trying out my msgspec library [1]. It works out-of-the-box with attrs objects (as well as its own faster `Struct` types), while being ~10-15x faster than cattrs for encoding/decoding/validating JSON. The hope is it's easy to integrate msgspec with other tools (like attrs!) rather than forcing the user to rewrite code to fit the new validation/serialization framework. It may not fit every use case, but if msgspec works for you it should be generally an order-of-magnitude faster than other Python options. [1]: https://github.com/jcrist/msgspec https://github.com/jcrist/msgspec </blatant-evangelism>
  - willhoyle 3y ago
    This looks like exactly what I've been looking for. I just want strong typing, json <-> struct and validation. Seems like it ticks all the boxes + speed benefits which is always nice. I especially find it useful that I can use messagepack for internal service chatter but still support json for external stuff and dump astuple to sqlite.
- eyegor 3y ago
  Depending on how far in you are, starlite/litestar has good documentation and offers another "batteries included" framework. Performance wise it's about the same and the stack is about the same. Fastapi suffers from the "one solo dev in Nebraska" paradigm (check out open prs and old tickets). For me the main draw of litestar is the batteries + better docs + more active development with multiple developers vs most other python web frameworks.
  - jammycrisp 3y ago
    +1 for litestar[1]. The higher bus-factor is nice, and I like that they're working to embrace a wider set of technologies than just pydantic. The framework currently lets you model objects using msgspec[2] (they actually use msgspec for all serialization), pydantic, or attrs[3], and the upcoming release adds some new mechanisms for handling additional types. I really appreciate the flexibility in modeling APIs; not everything fits well into a pydantic shaped box. [1]: https://litestar.dev/ https://litestar.dev/ [2]: https://github.com/jcrist/msgspec https://github.com/jcrist/msgspec [3]: https://www.attrs.org/en/stable/ https://www.attrs.org/en/stable/
  - nerdponx 3y ago
    I haven't heard of Starlite or Litestar before. Is one a fork of the other? Their documentation intro text is identical: > {Litestar|Starlite} is a powerful, flexible, highly performant, and opinionated ASGI framework, offering first class typing support and a full Pydantic integration. > > The {Litestar|Starlite} framework supports Plugins, ships with dependency injection, security primitives, OpenAPI schema generation, MessagePack, middlewares, and much more.
    - jammycrisp 3y ago
      starlite was the original name, it was recently renamed to litestar due to comments about how easily confused "starlette" and "starlite" are.
      - d0mine 3y ago
        data point in favor of the renaming: while reading GP, I've assumed that "starlite" is a typo for "starlette."
ramraj07 3y ago
Could you simplify your point? I was an ardent marshmallow user and when I finally switched to pydantic, it felt like I finally sat down in my life after standing forever. The documentation sounds good enough to me, but importantly the interface pydantic provides to define your json schema is the most elegant interface I’ve seen in any language and miles better than the mess marshmallow provided. For many of us especially in the SaaS side, speed of these operations is a distant third priority compared to ease of writing and understanding the code, and ensuring reliable less buggy code. The actual compute happens on a cluster with spark or snowflake anyway.
- nerdponx 3y ago
  There is no reference doc. The docs cover a lot of material in a small amount of space, buying important pieces of information and mixing up a large number of topics under unintuitive headlines. Reading the source code is occasionally necessary just to figure out how it all works. The API is a little weird, particularly around defining validators. The parameter name-matching is an "interesting" design choice. Accessing "values" as a dict[str,Any] is messy if you care about static typing, although I can understand why they did it. Furthermore, the behavior of validators and the exact sequence in which they run is not defined by the docs. It's not that hard to figure out, but it also might change at any time because there's no user contract. Attrs is significantly nicer in just about all respects here, especially their attention to detail in their extensive user guide and reference docs. Speaking of user contract, there's no clear separation between private and public. Without a reference doc it all looks like fair game, but without a reference doc it also might all change at any moment. Either you stick to the examples, or you're off doing a guess-and-check dance and hoping something doesn't break. Even with the Mypy plugin, I often have to write `if TYPE_CHECKING` all over any nontrivial Pydantic class consuming data from external sources. Variable annotations in Pydantic are fundamentally not PEP 484 type hints. That's fine, but it's confusing that they're almost the same, and, as above, it's almost entirely up to you to figure out how it all works, either by trial and error or by digging around in the issue tracker and StackOverflow. Ease of writing and reliability is precisely my big area of annoyance and concern. Speed of (de)serialization is comparatively unimportant (although I don't like the huge amount of overhead involved and I avoid using it in hot code paths). I also don't like using Pydantic-defined classes very much, because the actual init method signature is just *args, **kwargs, which doesn't work well with any tooling. It feels like being back in the Tornado & PyMongo dark ages where everything is dynamic or dynamically-generated and classes are just glorified hash tables. I agree that the JSONSchema integration is outstanding. BaseSettings is also a tremendous productivity improvement, I love that I can define a class and immediately get a proper app-wide config reading from both env vars and a dotenv file. I also like the default error messages that tell you exactly which field failed validation. I also like the validator system (once I figured out how it worked), respecting the order in which I define the validators as well as supporting validators that run before or after the default set of validators (pre=True and pre=False respectively). I was probably being a little too negative before, but my annoyance level with the developer-facing API and documentation remains high, and I will gladly jump to an Attrs-based alternative as soon as one exists.*