3 ms·
It’s not exactly the lazy parser you describe, but Sparser[1] builds filters to exclude json lines/files that can’t contain what you’re looking for, and only pa
by kristjansson 8y ago
It’s not exactly the lazy parser you describe, but Sparser[1] builds filters to exclude json lines/files that can’t contain what you’re looking for, and only parses those that might.
The Morning Paper’s writeup[2] from last year provides a good summary
[1]: http://www.vldb.org/pvldb/vol11/p1576-palkar.pdf http://www.vldb.org/pvldb/vol11/p1576-palkar.pdf
[2]: https://blog.acolyer.org/2018/08/20/filter-before-you-parse-faster-analytics-on-raw-data-with-sparser/ https://blog.acolyer.org/2018/08/20/filter-before-you-parse-...
- glangdale 8y agoThis work is somewhat orthogonal to ours as it assumes that you can locate JSON records without doing parsing; if I remember correctly, it groups JSON records as lines. If your JSON has been formatted to conform to this, I suppose it would be quite effective.