3 ms·

Are those alternatives faster according to benchmark? In my experience, the bottleneck is either: - JSON parsing and dumping; the solution for me is ORJSON, fa

Are those alternatives faster according to benchmark? In my experience, the bottleneck is either: - JSON parsing and dumping; the solution for me is ORJSON, fantastic wrapper to use fast JSON serialisation for most common fields, and also datetime. - Validation - if you choose to validate your data, pydantic can indeed be slow... But it's not Pydantic the problem, but the validation that you apply to your data.

progval 5y ago
> But it's not Pydantic the problem, but the validation that you apply to your data. Indeed, this kind of validation is usually based on 'isinstance', which is really slow in Python, because you often need to call it many times. More than once, I doubled of tripled the throughput of some data pipelines (not microbenchmarks) just by replacing 'isinstance' calls with something else. When you really need something like isinstance, type equality sometimes works and is much faster. For example, this works as a replacement for attrs-strict's type checking on a limited subset of types (non-generic classes, Any, Optional, Tuple, and Union): https://archive.softwareheritage.org/swh:1:cnt:7f4f1ea32eacef1317c49297430c15aa2dbea08c;origin=https://forge.softwareheritage.org/source/swh-model.git;visit=swh:1:snp:91a1f51040d40b57e164493ee95d474e0599b79e;anchor=swh:1:rev:a0f5436273ef6f5b62a388ae131ed5afa7287d00;path=/swh/model/model.py;lines=95-160 https://archive.softwareheritage.org/swh:1:cnt:7f4f1ea32eace... The downside is that you can't use subclasses of the specified types.