3 ms·
I’ve been using the ijson Python library for that - I have notes on that here: https://til.simonwillison.net/json/ijson-stream https://til.simonwillison.net/jso
by simonw 2y ago
I’ve been using the ijson Python library for that - I have notes on that here: https://til.simonwillison.net/json/ijson-stream https://til.simonwillison.net/json/ijson-stream
- Yanael 2y agoNice, it looks like a good library to build on top of. I like the available events: start_map, end_map, etc. I did try a library in JS that had similar ones, but it lacked the granularity to cover all use cases for individual fields instead of an entire item. I'll keep a note of this one if I do Python JSON streaming.
- jackmpcollins 2y agoPydantic also have support for parsing partial JSON. https://docs.pydantic.dev/latest/concepts/json/#partial-json-parsing https://docs.pydantic.dev/latest/concepts/json/#partial-json... from pydantic_core import from_json partial_json_data = '["aa", "bb", "c' result = from_json(partial_json_data, allow_partial=True) print(result) #> ['aa', 'bb'] You can also use their `jiter` package directly if you don't otherwise use pydantic. https://github.com/pydantic/jiter/tree/main/crates/jiter-python https://github.com/pydantic/jiter/tree/main/crates/jiter-pyt...
- simonw 2y agoThat's neat, I hadn't seen that. Docs were lacking so I submitted a PR: https://github.com/pydantic/jiter/pull/143 https://github.com/pydantic/jiter/pull/143