3 ms·
This would be more suitable to large json files if used with the `--stream` flag. Here's my take on it: jq -c --stream ' . as $in | select(length =
by sethish 7y ago
This would be more suitable to large json files if used with the `--stream` flag. Here's my take on it:
jq -c --stream '
. as $in
| select(length == 2)
| (
$in[0] | map(
if type == "number"
then "[" + tostring + "]"
else "." + .
end
) | add
) + " = " + ($in[1] | tostring)'
Using `--stream` allows jq to start before parsing the entire json file. In my experience, a 700mb json file can take up 5gb of ram in either jq or python -m json.
- jolmg 7y agoYou're right about `--stream`, but you didn't need the variable assignment. Also, `-c`, besides the fact that it's not available in jq-1.5 which some people are using, is very pointless in this situation, since we're looking to output text, and `-c` is for outputting objects/arrays in a compact format. The fact that `-r` wasn't used causes jq to output the text encoded as json strings. So, instead of outputting: .movie.name = "Interstellar" .movie.year = 2014 .movie.is_released = true .movie.else = "Christopher Nolan" .movie.cast[0] = "Matthew McConaughey" .movie.cast[1] = "Anne Hathaway" .movie.cast[2] = "Jessica Chastain" .movie.cast[3] = "Bill Irwin" .movie.cast[4] = "Ellen \\\\ Burstyn" .movie.cast[5] = "Michael Caine" You're outputting: ".movie.name = Interstellar" ".movie.year = 2014" ".movie.is_released = true" ".movie.else = Christopher Nolan" ".movie.cast[0] = Matthew McConaughey" ".movie.cast[1] = Anne Hathaway" ".movie.cast[2] = Jessica Chastain" ".movie.cast[3] = Bill Irwin" ".movie.cast[4] = Ellen \\\\ Burstyn" ".movie.cast[5] = Michael Caine" Another point is how the strings at the right of the `=` are displayed. They should be quoted. The reason why they're not is because you piped the second element to `tostring` instead of `@json`. A better version of your suggestion would've been: jq -r --stream ' select(length > 1) | ( .[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | add ) + " = " + (.[1] | @json) ' The use of `length > 1` instead of `length == 2` is a minor point, but if a future version jq decides to sometimes put 3 elements in these arrays, your filter would ignore those when we're likely to also want those. `length > 1` ensures what we need, that there are at least the elements that we're going to be using, while `length == 2` might filter some of those out, even if it's not right now. Your use of `add` is neat, though. I wouldn't have thought of that.
- facorreia 7y agoThanks! I've made a gist out of it: https://gist.github.com/fernandoacorreia/4b67a41bbe227654868df5eb3ca311d7 https://gist.github.com/fernandoacorreia/4b67a41bbe227654868...