6 ms·

I love jq, but I also use JMESPath (especially with AWS CLI), yq (bundled with tomlq and xq as well), and dasel [2]. I also wish hclq [3] wasn't so dead! [0]:

I love jq, but I also use JMESPath (especially with AWS CLI), yq (bundled with tomlq and xq as well), and dasel [2]. I also wish hclq [3] wasn't so dead! [0]: https://jmespath.org/ https://jmespath.org/ [1]: https://kislyuk.github.io/yq/ https://kislyuk.github.io/yq/ [2]: https://github.com/TomWright/dasel https://github.com/TomWright/dasel [3]: https://hclq.sh/ https://hclq.sh/

rmwaite 3y ago
I really like the JMESPath interactive tutorial page (https://jmespath.org/tutorial.html https://jmespath.org/tutorial.html). It helped me when I was first learning the syntax and I still go to it if I run into a particularly weird syntax that throws me off.
js2 3y ago
Make JSON greppable! https://github.com/tomnomnom/gron https://github.com/tomnomnom/gron I've been using `jq` for years and I'm always able to cobble together what I need, but I have yet to find it intuitive and I'm rarely able to arrive at a solution of any complexity without spending a lot of time reading its documentation. I wish I found it easier to use. :-(
- lathiat 3y ago
  I also love gron, if nothing else to find the paths I need to use with jq later. But ChatGPT has genuinely solved my suffering writing jq, it does a pretty good job. It even almost replaces gron, if you feed it an exmaple json and ask for jq, it gives you something. It usually needs a little adjusting but it gets me 90% of the way there and saves me a bit of time. I rarely use it for much else but its a jq winner :)
  - cryptonector 3y ago
    jq can be used trivially to approach gron-like output: jq -r 'paths(scalars) as $p | getpath($p) | "\($p|join(".")) = \(.)"' See elsewhere in this subthread for a full gron implementation in jq.
    - nikau 3y ago
      'Trivially'
      - cryptonector 3y ago
        Start with `jq -c 'paths(scalars)` -- that's pretty trivial.
        sah2ed 3y ago
        I think parent is referring to the habit of technically inclined folks of using "trivially" similar to words like "simply" and "just" [0][1] in a way that assumes too much about what the reader already knows. [0] https://www.parkersoftware.com/blog/stop-using-simply-in-tech-instructions/ https://www.parkersoftware.com/blog/stop-using-simply-in-tec... [1] https://news.ycombinator.com/item?id=35759449 https://news.ycombinator.com/item?id=35759449
        cryptonector 3y ago
        Or maybe I'm trying to whet one's appetite for learning the thing by showing a relatively simple expression that demonstrates the power of the language.
- akoboldfrying 3y ago
  Looks handy, but I'd rather go the other way and extend grep (and diff, etc.) to also work on things that aren't restricted to be lines in a text file. The number of times I've needed to go through contortions for things that should be easy solved problems (e.g., grepping for patterns of line pairs, grepping for records that use a different delimiter when the record data itself could contain linefeed, etc.)...
  - js2 3y ago
    Tip for pairs of lines. Use grep -A, -B, or -C to emit lines surrounding the first line to match, then pipe that into a second grep (also with -A, -B or -C). e.g. grep -A1 foo | grep -B1 bar Will find a line with "foo" followed by a line with "bar" and emit both. Of course, it will also find a single line with both "foo" and "bar", so it's not perfect. This is a quick and dirty solution. Beyond that, break out sed and awk, or maybe the Practical Extraction and Report Language... it's really good at that stuff.
    - akoboldfrying 3y ago
      Thanks, it's a brittle hack as you acknowledge but at least there won't be false negatives. Funny how often I wind up crawling back to Perl. The problem here is literally that someone hardcoded "IT'S ALWAYS LINEFEED" into an algorithm that could work equally well with any record separator character -- in fact probably with any record separator regex. I notice there's now `grep -z` which is one small step towards sanity... but the fully general problem is so easy and so useful to solve it's exasperating. I guess I should stop complaining and submit a patch to grep to add a `--dont-use-linefeed-instead-use <arg>` option already.
- cryptonector 3y ago
  You can trivially approximate gron with jq, and you there are jq implementations of gron too, like https://gist.github.com/mkfmnn/7b637aa0ec9b0422b3b75ec181a132fc https://gist.github.com/mkfmnn/7b637aa0ec9b0422b3b75ec181a13... E.g., # gron gron "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" | fgrep commit.author json[0].commit.author = {}; json[0].commit.author.date = "2016-07-02T10:51:21Z"; json[0].commit.author.email = "mail@tomnomnom.com"; json[0].commit.author.name = "Tom Hudson"; # jq curl -L "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" | jq -r 'paths(scalars) as $p | getpath($p) | "\($p|join(".")|select(contains("commit.author"))) = \(.)"' 0.commit.author.name = Tom Hudson 0.commit.author.email = mail@tomnomnom.com 0.commit.author.date = 2022-04-13T14:23:37Z # jq with grep outside jq curl -L "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" | jq -r 'paths(scalars) as $p | getpath($p) | "\($p|join(".")) = \(.)"' | fgrep commit.author 0.commit.author.name = Tom Hudson 0.commit.author.email = mail@tomnomnom.com 0.commit.author.date = 2022-04-13T14:23:37Z With just a bit more work you can get it to output valid gron, and even to parse valid gron.
  - archgoon 3y ago
    [dead]
AndyKluger 3y ago
A couple more alternatives: https://github.com/kellyjonbrazil/jello https://github.com/kellyjonbrazil/jello https://github.com/wwkimball/yamlpath https://github.com/wwkimball/yamlpath
commotionfever 3y ago
on a similar note I wrote little tool to convert from many formats to many formats the biggest usecase for me is taking some csv, toml, xml, whatever and converting that to json so I can pipe to jq https://github.com/sentriz/rsl https://github.com/sentriz/rsl
- cryptonector 3y ago
  Nice!!
wraptile 3y ago
Another great alternative is JSONPath[1] which unfortunately not as widely supported and known despite being brilliant! It's inspired by XPath so it's very familiar instead of a complete new DSL. The killer feature imo is the recursive key lookup so you can write `people..address` and it'll find all "address" keys that descend from "people" anywhere in the JSON. It's by far my favorite parsing language for JSON and I wrote an introduction blog on how to use it in JSON dataset parsing [2] :) 1 - https://github.com/JSONPath-Plus/JSONPath https://github.com/JSONPath-Plus/JSONPath 2 - https://scrapfly.io/blog/parse-json-jsonpath-python/ https://scrapfly.io/blog/parse-json-jsonpath-python/
- cryptonector 3y ago
  FYI jq also has recursive traversal.

rmwaite 3y ago

I really like the JMESPath interactive tutorial page (https://jmespath.org/tutorial.html https://jmespath.org/tutorial.html). It helped me when I was first learning the syntax and I still go to it if I run into a particularly weird syntax that throws me off.

js2 3y ago

Make JSON greppable! https://github.com/tomnomnom/gron https://github.com/tomnomnom/gron I've been using `jq` for years and I'm always able to cobble together what I need, but I have yet to find it intuitive and I'm rarely able to arrive at a solution of any complexity without spending a lot of time reading its documentation. I wish I found it easier to use. :-(

lathiat 3y ago
I also love gron, if nothing else to find the paths I need to use with jq later. But ChatGPT has genuinely solved my suffering writing jq, it does a pretty good job. It even almost replaces gron, if you feed it an exmaple json and ask for jq, it gives you something. It usually needs a little adjusting but it gets me 90% of the way there and saves me a bit of time. I rarely use it for much else but its a jq winner :)
- cryptonector 3y ago
  jq can be used trivially to approach gron-like output: jq -r 'paths(scalars) as $p | getpath($p) | "\($p|join(".")) = \(.)"' See elsewhere in this subthread for a full gron implementation in jq.
  - nikau 3y ago
    'Trivially'
    - cryptonector 3y ago
      Start with `jq -c 'paths(scalars)` -- that's pretty trivial.
      - sah2ed 3y ago
        I think parent is referring to the habit of technically inclined folks of using "trivially" similar to words like "simply" and "just" [0][1] in a way that assumes too much about what the reader already knows. [0] https://www.parkersoftware.com/blog/stop-using-simply-in-tech-instructions/ https://www.parkersoftware.com/blog/stop-using-simply-in-tec... [1] https://news.ycombinator.com/item?id=35759449 https://news.ycombinator.com/item?id=35759449
        cryptonector 3y ago
        Or maybe I'm trying to whet one's appetite for learning the thing by showing a relatively simple expression that demonstrates the power of the language.
akoboldfrying 3y ago
Looks handy, but I'd rather go the other way and extend grep (and diff, etc.) to also work on things that aren't restricted to be lines in a text file. The number of times I've needed to go through contortions for things that should be easy solved problems (e.g., grepping for patterns of line pairs, grepping for records that use a different delimiter when the record data itself could contain linefeed, etc.)...
- js2 3y ago
  Tip for pairs of lines. Use grep -A, -B, or -C to emit lines surrounding the first line to match, then pipe that into a second grep (also with -A, -B or -C). e.g. grep -A1 foo | grep -B1 bar Will find a line with "foo" followed by a line with "bar" and emit both. Of course, it will also find a single line with both "foo" and "bar", so it's not perfect. This is a quick and dirty solution. Beyond that, break out sed and awk, or maybe the Practical Extraction and Report Language... it's really good at that stuff.
  - akoboldfrying 3y ago
    Thanks, it's a brittle hack as you acknowledge but at least there won't be false negatives. Funny how often I wind up crawling back to Perl. The problem here is literally that someone hardcoded "IT'S ALWAYS LINEFEED" into an algorithm that could work equally well with any record separator character -- in fact probably with any record separator regex. I notice there's now `grep -z` which is one small step towards sanity... but the fully general problem is so easy and so useful to solve it's exasperating. I guess I should stop complaining and submit a patch to grep to add a `--dont-use-linefeed-instead-use <arg>` option already.
cryptonector 3y ago
You can trivially approximate gron with jq, and you there are jq implementations of gron too, like https://gist.github.com/mkfmnn/7b637aa0ec9b0422b3b75ec181a132fc https://gist.github.com/mkfmnn/7b637aa0ec9b0422b3b75ec181a13... E.g., # gron gron "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" | fgrep commit.author json[0].commit.author = {}; json[0].commit.author.date = "2016-07-02T10:51:21Z"; json[0].commit.author.email = "mail@tomnomnom.com"; json[0].commit.author.name = "Tom Hudson"; # jq curl -L "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" | jq -r 'paths(scalars) as $p | getpath($p) | "\($p|join(".")|select(contains("commit.author"))) = \(.)"' 0.commit.author.name = Tom Hudson 0.commit.author.email = mail@tomnomnom.com 0.commit.author.date = 2022-04-13T14:23:37Z # jq with grep outside jq curl -L "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" | jq -r 'paths(scalars) as $p | getpath($p) | "\($p|join(".")) = \(.)"' | fgrep commit.author 0.commit.author.name = Tom Hudson 0.commit.author.email = mail@tomnomnom.com 0.commit.author.date = 2022-04-13T14:23:37Z With just a bit more work you can get it to output valid gron, and even to parse valid gron.
- archgoon 3y ago
  [dead]

AndyKluger 3y ago

A couple more alternatives: https://github.com/kellyjonbrazil/jello https://github.com/kellyjonbrazil/jello https://github.com/wwkimball/yamlpath https://github.com/wwkimball/yamlpath

commotionfever 3y ago

on a similar note I wrote little tool to convert from many formats to many formats the biggest usecase for me is taking some csv, toml, xml, whatever and converting that to json so I can pipe to jq https://github.com/sentriz/rsl https://github.com/sentriz/rsl

cryptonector 3y ago
Nice!!

wraptile 3y ago

Another great alternative is JSONPath[1] which unfortunately not as widely supported and known despite being brilliant! It's inspired by XPath so it's very familiar instead of a complete new DSL. The killer feature imo is the recursive key lookup so you can write `people..address` and it'll find all "address" keys that descend from "people" anywhere in the JSON. It's by far my favorite parsing language for JSON and I wrote an introduction blog on how to use it in JSON dataset parsing [2] :) 1 - https://github.com/JSONPath-Plus/JSONPath https://github.com/JSONPath-Plus/JSONPath 2 - https://scrapfly.io/blog/parse-json-jsonpath-python/ https://scrapfly.io/blog/parse-json-jsonpath-python/

cryptonector 3y ago
FYI jq also has recursive traversal.