3 ms·
One of the components in our project was churning through thousands of JSONs per second - deserializing, transforming and serializing them. These JSONs represe
by gukoff 6y ago
One of the components in our project was churning through thousands of JSONs per second - deserializing, transforming and serializing them.
These JSONs represented the flight information. They included multiple datetimes, such as the scheduled departure/arrival time and the real departure/arrival time of a flight.
The first bottleneck was JSON deserializarion/serializarion. At that time we solved it with ujson, and now there's the even more performant orjson.
The second bottleneck happened to be datetime deserializarion. And we solved it with ciso8601 - luckily, these datetimes were in ISO8601. But this bottleneck later repeatedly occured in the other components and became an inspiration to write dtparse :)
- sillysaurusx 6y agoWow, orjson is amazing. It even serializes numpy arrays. Thanks!
- delduca 6y ago`pysimdjson` is even better!