4 ms·

Would probably be shorter if you used `tostream`

jolmg 7y ago
You're right. Good tip: jq -r ' tostream | select(length > 1) | ( .[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | join("") ) + " = " + (.[1] | @json) ' EDIT: For those who want to save this script, you can put just the jq code in an executable file with the shebang: #!/usr/bin/jq -rf
- hyperpallium 7y ago
  Appending | to each line and a last line . is a jq program that reproduces the original json. jq -r ' ( tostream | select(length > 1) | ( .[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | join("") ) + " = " + (.[1] | @json) + " |" ), "." '
  - jolmg 7y ago
    That's insane. Here I was thinking about how much more challenging it would be to parse and reconstruct the object from jq, and you got the idea to take advantage of the syntax similarity to parse it as jq code itself. Nice. And so, that means the inverse of the jq code I posted would simply be: ( jq "$(sed 's/$/ |/;$a.')" <<< '{}' ) As in: catj example.json \ | ( jq "$(sed 's/$/ |/;$a.')" <<< '{}' ) \ > original.json
    - hyperpallium 7y ago
      And a nice example of the path form being amenable to unix tools. BTW the input json can be null, so -n works (also using process substitution): jq -nf <(sed 's/$/ |/;$a.')
    - sdegutis 7y ago
      Everything you guys have posted in this thread looks like pure ninja magic to me.
      - cmonnow 7y ago
        looks like ancient Greek for us plebs
    - hyperpallium 7y ago
      Parsing is awkward in jq, but setpath(PATHS; VALUE) will create necessary structure. PATHS uses the array form, like ["movie", "cast", 5] not .movie.cast[5]. Since 1.5, jq has PCRE regex, so could remove ], and separate by . and [.
  - deleted 7y ago
    [deleted]
- sethish 7y ago
  This would be more suitable to large json files if used with the `--stream` flag. Here's my take on it: jq -c --stream ' . as $in | select(length == 2) | ( $in[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | add ) + " = " + ($in[1] | tostring)' Using `--stream` allows jq to start before parsing the entire json file. In my experience, a 700mb json file can take up 5gb of ram in either jq or python -m json.
  - jolmg 7y ago
    You're right about `--stream`, but you didn't need the variable assignment. Also, `-c`, besides the fact that it's not available in jq-1.5 which some people are using, is very pointless in this situation, since we're looking to output text, and `-c` is for outputting objects/arrays in a compact format. The fact that `-r` wasn't used causes jq to output the text encoded as json strings. So, instead of outputting: .movie.name = "Interstellar" .movie.year = 2014 .movie.is_released = true .movie.else = "Christopher Nolan" .movie.cast[0] = "Matthew McConaughey" .movie.cast[1] = "Anne Hathaway" .movie.cast[2] = "Jessica Chastain" .movie.cast[3] = "Bill Irwin" .movie.cast[4] = "Ellen \\\\ Burstyn" .movie.cast[5] = "Michael Caine" You're outputting: ".movie.name = Interstellar" ".movie.year = 2014" ".movie.is_released = true" ".movie.else = Christopher Nolan" ".movie.cast[0] = Matthew McConaughey" ".movie.cast[1] = Anne Hathaway" ".movie.cast[2] = Jessica Chastain" ".movie.cast[3] = Bill Irwin" ".movie.cast[4] = Ellen \\\\ Burstyn" ".movie.cast[5] = Michael Caine" Another point is how the strings at the right of the `=` are displayed. They should be quoted. The reason why they're not is because you piped the second element to `tostring` instead of `@json`. A better version of your suggestion would've been: jq -r --stream ' select(length > 1) | ( .[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | add ) + " = " + (.[1] | @json) ' The use of `length > 1` instead of `length == 2` is a minor point, but if a future version jq decides to sometimes put 3 elements in these arrays, your filter would ignore those when we're likely to also want those. `length > 1` ensures what we need, that there are at least the elements that we're going to be using, while `length == 2` might filter some of those out, even if it's not right now. Your use of `add` is neat, though. I wouldn't have thought of that.

You're right. Good tip: jq -r ' tostream | select(length > 1) | ( .[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | join("") ) + " = " + (.[1] | @json) ' EDIT: For those who want to save this script, you can put just the jq code in an executable file with the shebang: #!/usr/bin/jq -rf

hyperpallium 7y ago
Appending | to each line and a last line . is a jq program that reproduces the original json. jq -r ' ( tostream | select(length > 1) | ( .[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | join("") ) + " = " + (.[1] | @json) + " |" ), "." '
- jolmg 7y ago
  That's insane. Here I was thinking about how much more challenging it would be to parse and reconstruct the object from jq, and you got the idea to take advantage of the syntax similarity to parse it as jq code itself. Nice. And so, that means the inverse of the jq code I posted would simply be: ( jq "$(sed 's/$/ |/;$a.')" <<< '{}' ) As in: catj example.json \ | ( jq "$(sed 's/$/ |/;$a.')" <<< '{}' ) \ > original.json
  - hyperpallium 7y ago
    And a nice example of the path form being amenable to unix tools. BTW the input json can be null, so -n works (also using process substitution): jq -nf <(sed 's/$/ |/;$a.')
  - sdegutis 7y ago
    Everything you guys have posted in this thread looks like pure ninja magic to me.
    - cmonnow 7y ago
      looks like ancient Greek for us plebs
  - hyperpallium 7y ago
    Parsing is awkward in jq, but setpath(PATHS; VALUE) will create necessary structure. PATHS uses the array form, like ["movie", "cast", 5] not .movie.cast[5]. Since 1.5, jq has PCRE regex, so could remove ], and separate by . and [.
- deleted 7y ago
  [deleted]
sethish 7y ago
This would be more suitable to large json files if used with the `--stream` flag. Here's my take on it: jq -c --stream ' . as $in | select(length == 2) | ( $in[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | add ) + " = " + ($in[1] | tostring)' Using `--stream` allows jq to start before parsing the entire json file. In my experience, a 700mb json file can take up 5gb of ram in either jq or python -m json.
- jolmg 7y ago
  You're right about `--stream`, but you didn't need the variable assignment. Also, `-c`, besides the fact that it's not available in jq-1.5 which some people are using, is very pointless in this situation, since we're looking to output text, and `-c` is for outputting objects/arrays in a compact format. The fact that `-r` wasn't used causes jq to output the text encoded as json strings. So, instead of outputting: .movie.name = "Interstellar" .movie.year = 2014 .movie.is_released = true .movie.else = "Christopher Nolan" .movie.cast[0] = "Matthew McConaughey" .movie.cast[1] = "Anne Hathaway" .movie.cast[2] = "Jessica Chastain" .movie.cast[3] = "Bill Irwin" .movie.cast[4] = "Ellen \\\\ Burstyn" .movie.cast[5] = "Michael Caine" You're outputting: ".movie.name = Interstellar" ".movie.year = 2014" ".movie.is_released = true" ".movie.else = Christopher Nolan" ".movie.cast[0] = Matthew McConaughey" ".movie.cast[1] = Anne Hathaway" ".movie.cast[2] = Jessica Chastain" ".movie.cast[3] = Bill Irwin" ".movie.cast[4] = Ellen \\\\ Burstyn" ".movie.cast[5] = Michael Caine" Another point is how the strings at the right of the `=` are displayed. They should be quoted. The reason why they're not is because you piped the second element to `tostring` instead of `@json`. A better version of your suggestion would've been: jq -r --stream ' select(length > 1) | ( .[0] | map( if type == "number" then "[" + tostring + "]" else "." + . end ) | add ) + " = " + (.[1] | @json) ' The use of `length > 1` instead of `length == 2` is a minor point, but if a future version jq decides to sometimes put 3 elements in these arrays, your filter would ignore those when we're likely to also want those. `length > 1` ensures what we need, that there are at least the elements that we're going to be using, while `length == 2` might filter some of those out, even if it's not right now. Your use of `add` is neat, though. I wouldn't have thought of that.