4 ms·
Or just dump pydantic and use msgspec instead: https://jcristharif.com/msgspec/ https://jcristharif.com/msgspec/
by m_ke 1y ago
Or just dump pydantic and use msgspec instead: https://jcristharif.com/msgspec/ https://jcristharif.com/msgspec/
- itamarst 1y agomsgspec is much more memory efficient out of the box, yes. Also quite fast.
- mbb70 1y agoA great feature of pydantic are the validation hooks that let you intercept serialization/deserialization of specific fields and augment behavior. For example if you are querying a DB that returns a column as a JSON string, trivial with Pydantic to json parse the column are part of deser with an annotation. Pydantic is definitely slower and not a 'zero cost abstraction', but you do get a lot for it.
- jtmcivor 1y agoOne approach to do that in msgspec is described here https://github.com/jcrist/msgspec/issues/375#issuecomment-1520301756 https://github.com/jcrist/msgspec/issues/375#issuecomment-15...
- deleted 1y ago[deleted]
- aitchnyu 1y agoCan it do incremental parsing? Cant tell from a brief look.
- jtmcivor 1y agoIIUC: * You still need to load all the bytes into memory before passing to msgspec decoding * You can decode a subset of fields, which is really helpful * Reusing msgspec decoders saves some cpu cycles https://jcristharif.com/msgspec/perf-tips.html#reuse-encoders-decoders https://jcristharif.com/msgspec/perf-tips.html#reuse-encoder... Slides 17, 18, 19 have an example of the first two points https://pythonspeed.com/pycon2025/slides/#17 https://pythonspeed.com/pycon2025/slides/#17