3 ms·

I would love to see what jq looks like on something like a 1mil line Json vs this. In my experience jq syntax is fine and I've not ran into a performance issue

I would love to see what jq looks like on something like a 1mil line Json vs this. In my experience jq syntax is fine and I've not ran into a performance issue on any one file but I seem to see a jq clone every few months on here so someone seems to need that, or maybe it's just the new volume slider problem who knows.

justinsaccount 4y ago
jq performance is pretty terrible. Here I'm going to do something super simple like pull out a single field out of a large log file: $ wc -l big.log 979400 big.log $ du -hs big.log 570M big.log `count` is a small program that counts lines on stdin. like `sort|uniq -c |sort -n` jq takes 12 seconds: $ time cat big.log |jq -cr .method |~/bin/count 848000 GET 94800 POST 34000 HEAD 2400 OPTIONS 200 null real 0m12.381s user 0m12.427s sys 0m0.333s my tool takes .5 seconds $ time cat big.log |~/bin/jj method |~/bin/count 848000 GET 94800 POST 34000 HEAD 2400 OPTIONS 200 real 0m0.466s user 0m0.512s sys 0m0.198s `jj` is a little tool I wrote that uses https://github.com/buger/jsonparser https://github.com/buger/jsonparser