3 ms·

Python is so much harder to process JSON data in than jq, that that is how I got into working with and on jq almost a decade ago.

chatmasta 3y ago
Yeah, Python is like 10-20x the number of lines required to do the same thing as jq (especially with the boilerplate of consuming stdin), but that's also why it's more readable. But generally I agree - I would choose jq over some weird bash/python hybrid most of the time. I just wish it was more immediately readable.
- lathiat 3y ago
  I've found Ruby much nicer for writing dirty parsing logic like this in a "real" language, it lets you be more terse and "DRY" than Python. Which in bigger software projects doesn't hurt me as much but when I'm primarily trying to write something that otherwise would be well handled by SQL or JQ I found Ruby the better middleground for me.
- cryptonector 3y ago
  Simple jq programs are easy to read because simple jq programs are just path expressions, and the jq language is optimized to make path expressions easy to read. Path expressions like .[].commit | select(.author == "Tom Hudson") which basically says "find all commits by Tom Hudson" in the input. `.[]` iterates all the values in its input (whether the input be an array or an object). `.commit` gets the value of the "commit" key in the input object. You concatenate path expressions with `|`, and array/object index expressions you can just concatenate w/o `|`, so `.[]` and `.commit` can be `.[] | .commit` and also `.[].commit`. Calls to functions like `select()` whose bodies are path expressions are.. also path expressions. Perhaps the most brilliant thing about jq is that you can assign to arbitrarily complex path expressions, so you can: (.[].commit | select(.author == "Tom Hudson")) = "Anon" The syntax is strange probably because of this trying to make path expressions so trivial and readable. jq programs get hard to read mainly when you go beyond path expressions, especially when you start doing reductions. The problem is that it resembles point free programming in Haskell, which is really not for everyone. The other thing is that jq is very much a functional programming language, and that takes getting used to.
  - wrs 3y ago
    Also, here’s something that seems not widely appreciated: You can write super clever unreadable one-long-line jq programs embedded in bash scripts (I hear you on the point-free thing), or you can write jq programs that live in their own files, with multiple lines, indentation, comments, and intermediate assignments to variables with readable names. I recommend the latter!
    - flakes 3y ago
      Wait, really? I had no idea you could do that. Might have to try that next time Im tempted to break out python or node for a bash script.
      - wrs 3y ago
        I found a random example on GitHub for you. Search `path:jq$` for more. https://github.com/flox/flox/blob/019095f8bc40e49abc8e5cd0b1094e4c89a74f38/flox-bash/lib/diff-catalogs.jq#L1 https://github.com/flox/flox/blob/019095f8bc40e49abc8e5cd0b1...
  - rtpg 3y ago
    data = json.load(sys.stdin) commits = [elt.commit for elt in data if elt.commit.author = "Tom Hudson"] json.dump(commits, sys.stdout) Definitely not as straightforward... would be nice to have a bit more affordances for path expressions in Python.
    - dragonwriter 3y ago
      That doesn’t quite work, because JSON objects are parsed to Python dicts, not Python objects with properties, so it would be: data = json.load(sys.stdin) commits = [ e["commit"] for e in data if e["commit"]["author"] == "Tom Hudson" ] json.dump(commits, sys.stdout)
      - rtpg 3y ago
        Oops, yep totally. Even more futzy! Think if I was doing this a lot I'd totally pull out one of those "dict wrappers that allow for attr-based access" that lots of projects end up writing for whatever reason

chatmasta 3y ago

Yeah, Python is like 10-20x the number of lines required to do the same thing as jq (especially with the boilerplate of consuming stdin), but that's also why it's more readable. But generally I agree - I would choose jq over some weird bash/python hybrid most of the time. I just wish it was more immediately readable.

lathiat 3y ago
I've found Ruby much nicer for writing dirty parsing logic like this in a "real" language, it lets you be more terse and "DRY" than Python. Which in bigger software projects doesn't hurt me as much but when I'm primarily trying to write something that otherwise would be well handled by SQL or JQ I found Ruby the better middleground for me.
cryptonector 3y ago
Simple jq programs are easy to read because simple jq programs are just path expressions, and the jq language is optimized to make path expressions easy to read. Path expressions like .[].commit | select(.author == "Tom Hudson") which basically says "find all commits by Tom Hudson" in the input. `.[]` iterates all the values in its input (whether the input be an array or an object). `.commit` gets the value of the "commit" key in the input object. You concatenate path expressions with `|`, and array/object index expressions you can just concatenate w/o `|`, so `.[]` and `.commit` can be `.[] | .commit` and also `.[].commit`. Calls to functions like `select()` whose bodies are path expressions are.. also path expressions. Perhaps the most brilliant thing about jq is that you can assign to arbitrarily complex path expressions, so you can: (.[].commit | select(.author == "Tom Hudson")) = "Anon" The syntax is strange probably because of this trying to make path expressions so trivial and readable. jq programs get hard to read mainly when you go beyond path expressions, especially when you start doing reductions. The problem is that it resembles point free programming in Haskell, which is really not for everyone. The other thing is that jq is very much a functional programming language, and that takes getting used to.
- wrs 3y ago
  Also, here’s something that seems not widely appreciated: You can write super clever unreadable one-long-line jq programs embedded in bash scripts (I hear you on the point-free thing), or you can write jq programs that live in their own files, with multiple lines, indentation, comments, and intermediate assignments to variables with readable names. I recommend the latter!
  - flakes 3y ago
    Wait, really? I had no idea you could do that. Might have to try that next time Im tempted to break out python or node for a bash script.
    - wrs 3y ago
      I found a random example on GitHub for you. Search `path:jq$` for more. https://github.com/flox/flox/blob/019095f8bc40e49abc8e5cd0b1094e4c89a74f38/flox-bash/lib/diff-catalogs.jq#L1 https://github.com/flox/flox/blob/019095f8bc40e49abc8e5cd0b1...
- rtpg 3y ago
  data = json.load(sys.stdin) commits = [elt.commit for elt in data if elt.commit.author = "Tom Hudson"] json.dump(commits, sys.stdout) Definitely not as straightforward... would be nice to have a bit more affordances for path expressions in Python.
  - dragonwriter 3y ago
    That doesn’t quite work, because JSON objects are parsed to Python dicts, not Python objects with properties, so it would be: data = json.load(sys.stdin) commits = [ e["commit"] for e in data if e["commit"]["author"] == "Tom Hudson" ] json.dump(commits, sys.stdout)
    - rtpg 3y ago
      Oops, yep totally. Even more futzy! Think if I was doing this a lot I'd totally pull out one of those "dict wrappers that allow for attr-based access" that lots of projects end up writing for whatever reason