4 ms·
It's very possible it could be faster; jq seems to actually be fairly unoptimized. This implementation in OCaml was featured on HN a while back and it trashes t
by brundolf 4y ago
It's very possible it could be faster; jq seems to actually be fairly unoptimized. This implementation in OCaml was featured on HN a while back and it trashes the original jq in performance: https://github.com/davesnx/query-json https://github.com/davesnx/query-json
After seeing that one I did my own (less-complete) version in Rust and managed to squeeze out even more performance in the operations it supports: https://github.com/brundonsmith/jqr https://github.com/brundonsmith/jqr
- cerved 4y agojq is particularly bad at large stream progressing
- oever 4y agoWorking with large json files is hard to parallelize. Just filtering the objects in a root array can take very long. jqr and gojq both die with OOM when running on large files like https://dumps.wikimedia.org/wikidatawiki/entities/latest-all.json.gz https://dumps.wikimedia.org/wikidatawiki/entities/latest-all... A fast tool to split a json file like that into a format with one json file per line would already help a lot.
- brundolf 4y agoMine can :)