4 ms·

> It's not as arcane as it seems. The issue with jq is that I use it maybe once a month, or even less. The syntax is "arcane enough" that I keep forgetting how

> It's not as arcane as it seems. The issue with jq is that I use it maybe once a month, or even less. The syntax is "arcane enough" that I keep forgetting how to use it because I use it so sporadically. In comparison awk – which I also don't use that often – has a much easier syntax that I can mostly remember. Not entirely convinced by the zq syntax either though; it also seems "arcane enough" that I would keep forgetting it.

taude 5y ago
Same issue. However, I do successfully rely on using ctrl-r a lot to search prior invoked commands. And have a few core aliases that I've cobbled together....
- rgoodwintx 5y ago
  Here because.... I didn't know of ctrl-R. What a life changer (although I had an alias for "hg" to "history | grep" :) )
  - kalev 5y ago
    Please check FZF [1] and it’s integration with ctrl-r. It’s a huge productivity boost and I cannot live without it. [1] https://github.com/junegunn/fzf https://github.com/junegunn/fzf
    - lillecarl 5y ago
      There's also McFly[1] that does interactive history search. [1]: https://github.com/cantino/mcfly https://github.com/cantino/mcfly
      - mathfailure 5y ago
        This sounds quite awesome, it's a pity that the realization is quite 'alien' to unix way of things. IMO shell commands history should be a database by default, not just a single file that gets auto-appended with both awesome commands and crap you don't really want to get stored for later. Thanks for linking this project, will try it, it may be a game changer.
    - rgoodwintx 5y ago
      I think I actually HAVE fzf installed, thus my first experience w/ Ctrl-R was even better! I really haven't fully grok'ed fzf yet though, probably should look up some guides.
hiram112 5y ago
Bingo. There are at least a dozen tools and languages and syntaxes that I've used sporadically over the years - awk, sed, bash, Mongo, perl, etc. I don't use them often enough to remember exactly how they work, and so I always have to spend a few hours reviewing manuals or old code repos or an O'Reilly book. But if I do end up using it for a few days in a row, it starts to make sense, and I improve each time I use it. But not with jq. It just does not make sense to my brain, no matter how many times I've had to use it. Every single time I need to use it, it requires finding some Stack Exchange or blog and just copying and pasting. Even after seeing the solution, rarely do I then really understand why or how it works. Nor can I often take that knowledge and apply it to similar problems. About the only other syntax or language that gives me such problems is Elastic Search DSL.
- silon42 5y ago
  Same for me... everytime I have to lookup the basics... and I love awk,perl and xpath/xslt.
ts0000 5y ago
Interesting, for me it's the exact opposite. I've tried a couple of times to get into awk, but still find the syntax arcane.
- Beltalowda 5y ago
  I don't know; I wouldn't presume to tell you what you do or don't find arcane, but once I understood the somewhat unusual flow of awk ("for every line, check if the line matches this condition, and if it does run this block of code") I found it's quite easy to work with. It's "arcane" in the sense that it has an implicit loop and that it's a specialized language for a very limited class of problems, but I found that for this limited class of problem it's surprisingly effective.
  - dotancohen 5y ago
    > an implicit loop As an occasional awk user, I'd love if you expand on this. Maybe it will help clear things up for me. You're not referring to the fact that awk operates on every line independently, are you?
    - Beltalowda 5y ago
      My mental image of awk has always been something along these lines: for line in readfile() for block in script: if block.match(line) run_block(block) end endfor endfor Where the "for line in readfile()" is the "implicit loop", and the blocks are the "condition { .. }" blocks. The actual flow is a little bit more complex and has some exceptions e.g. (BEGIN/END), but this is about the gist of it.
      - dotancohen 5y ago
        Thanks. Yes, I agree that my mental image is pretty much the same but it's nice to see it expressed in Python modulo end keywords ))
    - Brentward 5y ago
laurent123456 5y ago
I wonder if someone tried to use plain JS as a filtering language? It would be more verbose but it would be easy to remember. For example: [1,2,3] | js "out = 0; for (const n of this) out += n" That would print "6". `out` would be a special variable you write to to print the result, and `this` would be the input.
- Beltalowda 5y ago
  A few of the tools listed here seem to work like that, or roughly similar: https://ilya-sher.org/2018/04/10/list-of-json-tools-for-command-line/ https://ilya-sher.org/2018/04/10/list-of-json-tools-for-comm... I didn't check any of them out though.
- mechanicalpulse 5y ago
  I've used trentm's json (formerly known as jsontool) package from npm as my default tool for command-line manipulation of JSON for many years now. It provides CLI arguments for passing JavaScript code for filtering and executing on input. I have resisted investing the time into becoming fluent in jq because I've found that many of the common use cases I have are readily handled by jsontool. https://www.npmjs.com/package/json https://www.npmjs.com/package/json Edit: added more information
- lgas 5y ago
  My hope was to one day add JS eval support to https://github.com/SuperpowersCorp/refactorio https://github.com/SuperpowersCorp/refactorio but as you can tell by the timestamps I haven't found any time to work on it in the last 4 years.
- rane 5y ago
  Not quite that, but ramda-cli[1] which I've created solves this problem, at least for me, by offering the familiar set of functions from Ramda, and you can create pipelines with those to do operations on your data. [1]: https://github.com/raine/ramda-cli https://github.com/raine/ramda-cli
- anitil 5y ago
  That's a really interesting suggestion, similar to how AWK uses $0, $1 etc.
ar_lan 5y ago
This is ironic - I use `awk` so infrequently, I have no idea how to use it without reading its man page or using Google. But I use `jq` often and find it simple.
zeroimpl 5y ago
I use both awk and jq infrequently enough that I tend to struggle with anything non-trivial. I think zq would be the same. > Not entirely convinced by the zq syntax either though; it also seems "arcane enough" that I would keep forgetting it. I think this is the main thing. I’d prefer a streamlined CLI tool where you passed in some JS code and it’d just run it on the input (with the same slurp/raw args as jq). Could just be npm with underscore.js autoimported.

taude 5y ago

Same issue. However, I do successfully rely on using ctrl-r a lot to search prior invoked commands. And have a few core aliases that I've cobbled together....

rgoodwintx 5y ago
Here because.... I didn't know of ctrl-R. What a life changer (although I had an alias for "hg" to "history | grep" :) )
- kalev 5y ago
  Please check FZF [1] and it’s integration with ctrl-r. It’s a huge productivity boost and I cannot live without it. [1] https://github.com/junegunn/fzf https://github.com/junegunn/fzf
  - lillecarl 5y ago
    There's also McFly[1] that does interactive history search. [1]: https://github.com/cantino/mcfly https://github.com/cantino/mcfly
    - mathfailure 5y ago
      This sounds quite awesome, it's a pity that the realization is quite 'alien' to unix way of things. IMO shell commands history should be a database by default, not just a single file that gets auto-appended with both awesome commands and crap you don't really want to get stored for later. Thanks for linking this project, will try it, it may be a game changer.
  - rgoodwintx 5y ago
    I think I actually HAVE fzf installed, thus my first experience w/ Ctrl-R was even better! I really haven't fully grok'ed fzf yet though, probably should look up some guides.

hiram112 5y ago

Bingo. There are at least a dozen tools and languages and syntaxes that I've used sporadically over the years - awk, sed, bash, Mongo, perl, etc. I don't use them often enough to remember exactly how they work, and so I always have to spend a few hours reviewing manuals or old code repos or an O'Reilly book. But if I do end up using it for a few days in a row, it starts to make sense, and I improve each time I use it. But not with jq. It just does not make sense to my brain, no matter how many times I've had to use it. Every single time I need to use it, it requires finding some Stack Exchange or blog and just copying and pasting. Even after seeing the solution, rarely do I then really understand why or how it works. Nor can I often take that knowledge and apply it to similar problems. About the only other syntax or language that gives me such problems is Elastic Search DSL.

silon42 5y ago
Same for me... everytime I have to lookup the basics... and I love awk,perl and xpath/xslt.

ts0000 5y ago

Interesting, for me it's the exact opposite. I've tried a couple of times to get into awk, but still find the syntax arcane.

Beltalowda 5y ago
I don't know; I wouldn't presume to tell you what you do or don't find arcane, but once I understood the somewhat unusual flow of awk ("for every line, check if the line matches this condition, and if it does run this block of code") I found it's quite easy to work with. It's "arcane" in the sense that it has an implicit loop and that it's a specialized language for a very limited class of problems, but I found that for this limited class of problem it's surprisingly effective.
- dotancohen 5y ago
  > an implicit loop As an occasional awk user, I'd love if you expand on this. Maybe it will help clear things up for me. You're not referring to the fact that awk operates on every line independently, are you?
  - Beltalowda 5y ago
    My mental image of awk has always been something along these lines: for line in readfile() for block in script: if block.match(line) run_block(block) end endfor endfor Where the "for line in readfile()" is the "implicit loop", and the blocks are the "condition { .. }" blocks. The actual flow is a little bit more complex and has some exceptions e.g. (BEGIN/END), but this is about the gist of it.
    - dotancohen 5y ago
      Thanks. Yes, I agree that my mental image is pretty much the same but it's nice to see it expressed in Python modulo end keywords ))
  - Brentward 5y ago

laurent123456 5y ago

I wonder if someone tried to use plain JS as a filtering language? It would be more verbose but it would be easy to remember. For example: [1,2,3] | js "out = 0; for (const n of this) out += n" That would print "6". `out` would be a special variable you write to to print the result, and `this` would be the input.

Beltalowda 5y ago
A few of the tools listed here seem to work like that, or roughly similar: https://ilya-sher.org/2018/04/10/list-of-json-tools-for-command-line/ https://ilya-sher.org/2018/04/10/list-of-json-tools-for-comm... I didn't check any of them out though.
mechanicalpulse 5y ago
I've used trentm's json (formerly known as jsontool) package from npm as my default tool for command-line manipulation of JSON for many years now. It provides CLI arguments for passing JavaScript code for filtering and executing on input. I have resisted investing the time into becoming fluent in jq because I've found that many of the common use cases I have are readily handled by jsontool. https://www.npmjs.com/package/json https://www.npmjs.com/package/json Edit: added more information
lgas 5y ago
My hope was to one day add JS eval support to https://github.com/SuperpowersCorp/refactorio https://github.com/SuperpowersCorp/refactorio but as you can tell by the timestamps I haven't found any time to work on it in the last 4 years.
rane 5y ago
Not quite that, but ramda-cli[1] which I've created solves this problem, at least for me, by offering the familiar set of functions from Ramda, and you can create pipelines with those to do operations on your data. [1]: https://github.com/raine/ramda-cli https://github.com/raine/ramda-cli
anitil 5y ago
That's a really interesting suggestion, similar to how AWK uses $0, $1 etc.

ar_lan 5y ago

This is ironic - I use `awk` so infrequently, I have no idea how to use it without reading its man page or using Google. But I use `jq` often and find it simple.

zeroimpl 5y ago

I use both awk and jq infrequently enough that I tend to struggle with anything non-trivial. I think zq would be the same. > Not entirely convinced by the zq syntax either though; it also seems "arcane enough" that I would keep forgetting it. I think this is the main thing. I’d prefer a streamlined CLI tool where you passed in some JS code and it’d just run it on the input (with the same slurp/raw args as jq). Could just be npm with underscore.js autoimported.