5 ms·
Pydantic
- Starlevel001 5y ago
- metadat 5y agoHN Guidelines state, among other things worth noting [1]: > Please don't complain about tangential annoyances—things like article or website formats, name collisions, or back-button breakage. They're too common to be interesting. [1] https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- Starlevel001 5y ago
- metadat 5y agoDear Starlevel, Sorry if I came across as unhelpful, my intent was actually the opposite - to inform and hopefully educate. I don't like wasting what time I have left on this earth bothering people with unwelcome or unappreciated feedback (though I accept this is always a risk when engaging, especially via Internet forums where tone is not discernable). Your argument would be strengthened if you added more explanation or insight into why projects using your disliked, cursed documentation format definitively suck :) Feedback is always welcome, I'm always looking to improve, learn how to be more persuasive, and avoid miscommunication. Sincerely, metadat@hn
- JPKab 5y agoWe use this extensively as probably half of the Python programmers on HN do. It's great, but there are plenty of alternatives out there as well, especially for performance. Our specific use case is IO limited so it's perfect for that.
- jamil7 5y agoCare to list some? I disabled it on a few endpoints of our backend because the performance was awful for GeoJSON.
- lazypenguin 5y agohttps://www.attrs.org/en/stable/ https://www.attrs.org/en/stable/ would be the main alternative
- johnwheeler 5y agoMarshmallow is another if you’re talking schemas, validation, and json serialization.
- hackandtrip 5y agoAre those alternatives faster according to benchmark? In my experience, the bottleneck is either: - JSON parsing and dumping; the solution for me is ORJSON, fantastic wrapper to use fast JSON serialisation for most common fields, and also datetime. - Validation - if you choose to validate your data, pydantic can indeed be slow... But it's not Pydantic the problem, but the validation that you apply to your data.
- progval 5y ago> But it's not Pydantic the problem, but the validation that you apply to your data. Indeed, this kind of validation is usually based on 'isinstance', which is really slow in Python, because you often need to call it many times. More than once, I doubled of tripled the throughput of some data pipelines (not microbenchmarks) just by replacing 'isinstance' calls with something else. When you really need something like isinstance, type equality sometimes works and is much faster. For example, this works as a replacement for attrs-strict's type checking on a limited subset of types (non-generic classes, Any, Optional, Tuple, and Union): https://archive.softwareheritage.org/swh:1:cnt:7f4f1ea32eacef1317c49297430c15aa2dbea08c;origin=https://forge.softwareheritage.org/source/swh-model.git;visit=swh:1:snp:91a1f51040d40b57e164493ee95d474e0599b79e;anchor=swh:1:rev:a0f5436273ef6f5b62a388ae131ed5afa7287d00;path=/swh/model/model.py;lines=95-160 https://archive.softwareheritage.org/swh:1:cnt:7f4f1ea32eace... The downside is that you can't use subclasses of the specified types.
- dang 5y agoPast related threads: PEP 563, PEP 649 and the future of pydantic and FastAPI - https://news.ycombinator.com/item?id=26826158 https://news.ycombinator.com/item?id=26826158 - April 2021 (150 comments) Show HN: Pydantic – Data validation using Python 3.6 type hinting - https://news.ycombinator.com/item?id=14477222 https://news.ycombinator.com/item?id=14477222 - June 2017 (27 comments)
- null_deref 5y agoUsing Pydantic, FastAPI and OpenAPI generator tool was a decision which increased my speed of writing code by a lot. Plus it didn't hurt, that FastAPI has an excellent documentation (highly recommend the part that describes using FastAPI with SQLAlchemy).
- BerislavLopac 5y agoOpenAPI generation is cool, but using the OpenAPI as the specification of even better. ;)
- hackandtrip 5y agoLove FastAPI for lot of concepts that it helped to make mainstream, such as typing-based-validation, but its documentation is lacking. I love the tutorials, but an API reference is just as important; I may not want to check out long tutorials about things I already read, but check a detailed description of each function would be super helpful - other frameworks such as Sanic have it. I think it's in the roadmap, kudos for that, and hope that it will get a few more maintainers to speed up the process
- oars 5y agoFantastic project, but if you're looking for speed this isn't it.
- bredren 5y agoWhat do you look to for a more balanced speed / feature set?
- Seirdy 5y agoA different language, most likely. I'd use Python for this if the work being done isn't too CPU-intensive and I can afford to make big tradeoffs. If you want to do data validation like this but for something with better performance while still retaining the benefits of a high-level GC'd language, then I'd try something like https://github.com/go-playground/validator https://github.com/go-playground/validator for Go.
- Attummm 5y agoTake a look at https://github.com/Attumm/Maat https://github.com/Attumm/Maat Maat is much faster then pydantic according to the benchmark of pydantic.
- johnwheeler 5y agoLooks atrocious from an ergonomics perspective. Who likes defining everything with python dictionaries over classes or type hints?
- mythrwy 5y agoPeople who want stuff to go fast?
- johnwheeler 5y agoOk, you go do that then
- 5y ago
- shroompasta 5y agoLooks great. I'm hesitant, however, since marshmallow-sqlalchemy provides full integration with your SQLAlchemy models, but pydantic-sqlalchemy only is for generating Pydantic models based on SQLAlchemy models, and it seems as if it's still experimental (Why does it have more stars? :thinking:) Otherwise, just between pydantic and marshmallow for straight up validations, it seems Pydantic is more legible and easier to use at first sight. Will switch if pydantic had full integration with SQLAlchemy. And for those of you looking down on me for using ORMs (yes, i know some of you exist), I use both raw SQL and SQLAlchemy. I find it multitudes easier to build models and deal with migrations in SQLAlchemy than writing scripts.
- smoe 5y agoThe creator of FastAPI and pydantic-sqlalchemy has recently released a new library: SQLModel. https://sqlmodel.tiangolo.com https://sqlmodel.tiangolo.com It is a thin layer on top of Pydantic and SQLAlchemy. I haven't used it yet, so can't speak out of experience, but I think it is basically exactly what you describe.
- shroompasta 5y agolooks promising. At first sight it seems like you still have to write a "schema" for the SQLModel based on your SQLALchemy model - so basically, two sources of truth. If you edit your SQLAlchemy model, you'll also have to edit your SQLModel. sqlalchemy-marshmallow allows you to build your schema based on your SQLAlchemy model. Other than that, I'm still somewhat intrigued. Thanks for the suggestion.
- fantod 5y agoI think SQLModel actually exists precisely so that you don't have to do that. According to [1]: > That class Hero is a SQLModel model... But at the same time, it is a SQLAlchemy model... And at the same time, it is also a Pydantic model [1] https://sqlmodel.tiangolo.com/#sqlalchemy-and-pydantic https://sqlmodel.tiangolo.com/#sqlalchemy-and-pydantic