2 ms·
The advantage over standard json is that this is trivial to parse incrementally, i.e. for a dataset that is too big to fit in memory. I also wouldn't recommend
by ma2rten 11y ago
The advantage over standard json is that this is trivial to parse incrementally, i.e. for a dataset that is too big to fit in memory.
I also wouldn't recommend you to reimplement a functionality that is already part of the standard library, especially because the standard library function is going to be much faster because it's implemented in C.
- Rangi42 11y agoThanks for that info. I didn't know the module was written in C.
- ma2rten 11y agoOnly a small subset of the python standard library is actually written in python.
- toomim 11y agoActually, the approach with commas and a wrapped brackets are equally trivial to parse incrementally. Just start by erasing the open "[", and then read one row at a time, just like you would do without the "[" and comma.
- smsm42 11y agoI would say any tool that outputs lists of things in JSON and there is reasonable suspicion that the output may be large, should do this. I.e. do one-thing-per-line with opening and closing brackets on separate lines. It is quite annoying to find oneself with huge dataset in JSON which was produced by a naive generator that doesn't do that - when you need to extract something like "first N elements that satisfy condition X" it is annoyingly more work that it should be. By doing this people that don't care still get valid JSON, people that do care save some time.