6 ms·

To a Man with `Jq`, Everything Looks Like JSON

48 points by ishandotpage 3y ago

mdaniel 3y ago
heh, wait until one discovers that gojq <https://github.com/itchyny/gojq#difference-to-jq https://github.com/itchyny/gojq#difference-to-jq> accepts yaml input; so, if you happen to have a bunch of _almost_ structured data then some light sprinkling can turn it into yaml and then you're back in the loving embrace of the jq transformation/mutation language :heart: $ cat some-vendor-nonsense.txt product id: 5 price: 3.14 product id: 6 price: 6.66 $ sed "s/^/ /; s/^ product/- product/" < some-vendor-nonsense.txt \ | gojq --yaml-input .
- lathiat 3y ago
  Oh, that is great.. I was using dasel to convert yaml to json first ha ha.
- madduci 3y ago
  Or just use "yq", if you need to parse yaml
  - mdaniel 3y ago
    You somehow thought that my problem was parsing yaml, when I thought I made it abundantly clear that it was jq's power that was my interest; yq has its own syntax and is not the jqlang that I use
- wwader 3y ago
  Did someone say let's represent structured data as json? a bit of shameless plug: https://github.com/wader/fq https://github.com/wader/fq :) It's using a fork of gojq btw!
  - mdaniel 3y ago
    Oh, I didn't know that; how cool! Have you asked itchyny about their interest in upstreaming the changes <https://github.com/itchyny/gojq/compare/main...wader:gojq:fq https://github.com/itchyny/gojq/compare/main...wader:gojq:fq>? they don't look completely out of alignment with gojq and it would make reasoning about "gojq version what?" in fq easier and the inverse merge of theirs versus your fork seems to have a substantial changelog entry https://github.com/wader/gojq/compare/fq...itchyny:gojq:main https://github.com/wader/gojq/compare/fq...itchyny:gojq:main
    - wwader 3y ago
      Yeap i've talked to itchyny quite a lot about various changes https://github.com/itchyny/gojq/issues/153 https://github.com/itchyny/gojq/issues/153 and also upstreamed quite a lot https://github.com/itchyny/gojq/issues?q=author%3Awader https://github.com/itchyny/gojq/issues?q=author%3Awader like custom iterators (to allow eval, own iterators and "empty" functions), query marshalling (query rewrite tricks) and a bunch of small things and bug fixes. But the largest change to add a JQValue interface is quite complex, other changes like extended literals is also a bit tricky. Hmm weird list of changes for https://github.com/wader/gojq/compare/fq...itchyny:gojq:main https://github.com/wader/gojq/compare/fq...itchyny:gojq:main but i guess it is because i haven't kept my main branch in sync. The fq branch should be based on latest gojq/main as of now. I usually try to rebase as quick as possible. Let me know if you have any other questions or want to help out! maybe email etc as i usually don't check HN comments replies that often :)
QuercusMax 3y ago
At some point in the last 25 years I learned "perl -pi -e" and "for x in [whatever]; do [something] $x; done" and I type those without thinking when I'm working in the shell. I haven't written any perl in probably 12 years, but I still use it for file manipulation.
deleted 3y ago
[deleted]
mpalmer 3y ago
jq has string interpolation too, so it can even get a little bit nicer: "<option value=\"/examples/\(.)/\">\(.)</option>"
- mdaniel 3y ago
  Yeah, but muscle memory bites me all the time and I put the backslash on the closing paren, too, because I'm so used to the regex usage of that syntax which needs them to match I also want to draw the reader's attention to the magic of |@uri <https://github.com/jqlang/jq/blob/jq-1.7/docs/content/manual/manual.yml#L2097 https://github.com/jqlang/jq/blob/jq-1.7/docs/content/manual...> for a bunch of cases, but doubly so in TFA's case where they're plugging strings into a URI context. Simple string concat often works great for "hello world", but the world is not always just hello, so one quick use of the filter and jq's got your back echo "the world's scary" | jq -Rr '"<option value=\"/examples/\(.|@uri)\">\(.)</value>"'
sam0x17 3y ago
> Back in the Bootstrap 3 days, there used to be a customizer. There is still a quite good bootstrap customizer at bootstrap.build btw ;)
rnemeth91 3y ago
If you'd like to use jq in any JetBrains IDE (e.g. IntelliJ), here's a plugin for it: https://plugins.jetbrains.com/plugin/23360-jqexpress https://plugins.jetbrains.com/plugin/23360-jqexpress

48 points by ishandotpage 3y ago

heh, wait until one discovers that gojq <https://github.com/itchyny/gojq#difference-to-jq https://github.com/itchyny/gojq#difference-to-jq> accepts yaml input; so, if you happen to have a bunch of _almost_ structured data then some light sprinkling can turn it into yaml and then you're back in the loving embrace of the jq transformation/mutation language :heart: $ cat some-vendor-nonsense.txt product id: 5 price: 3.14 product id: 6 price: 6.66 $ sed "s/^/ /; s/^ product/- product/" < some-vendor-nonsense.txt \ | gojq --yaml-input .

lathiat 3y ago
Oh, that is great.. I was using dasel to convert yaml to json first ha ha.
madduci 3y ago
Or just use "yq", if you need to parse yaml
- mdaniel 3y ago
  You somehow thought that my problem was parsing yaml, when I thought I made it abundantly clear that it was jq's power that was my interest; yq has its own syntax and is not the jqlang that I use
wwader 3y ago
Did someone say let's represent structured data as json? a bit of shameless plug: https://github.com/wader/fq https://github.com/wader/fq :) It's using a fork of gojq btw!
- mdaniel 3y ago
  Oh, I didn't know that; how cool! Have you asked itchyny about their interest in upstreaming the changes <https://github.com/itchyny/gojq/compare/main...wader:gojq:fq https://github.com/itchyny/gojq/compare/main...wader:gojq:fq>? they don't look completely out of alignment with gojq and it would make reasoning about "gojq version what?" in fq easier and the inverse merge of theirs versus your fork seems to have a substantial changelog entry https://github.com/wader/gojq/compare/fq...itchyny:gojq:main https://github.com/wader/gojq/compare/fq...itchyny:gojq:main
  - wwader 3y ago
    Yeap i've talked to itchyny quite a lot about various changes https://github.com/itchyny/gojq/issues/153 https://github.com/itchyny/gojq/issues/153 and also upstreamed quite a lot https://github.com/itchyny/gojq/issues?q=author%3Awader https://github.com/itchyny/gojq/issues?q=author%3Awader like custom iterators (to allow eval, own iterators and "empty" functions), query marshalling (query rewrite tricks) and a bunch of small things and bug fixes. But the largest change to add a JQValue interface is quite complex, other changes like extended literals is also a bit tricky. Hmm weird list of changes for https://github.com/wader/gojq/compare/fq...itchyny:gojq:main https://github.com/wader/gojq/compare/fq...itchyny:gojq:main but i guess it is because i haven't kept my main branch in sync. The fq branch should be based on latest gojq/main as of now. I usually try to rebase as quick as possible. Let me know if you have any other questions or want to help out! maybe email etc as i usually don't check HN comments replies that often :)

QuercusMax 3y ago

At some point in the last 25 years I learned "perl -pi -e" and "for x in [whatever]; do [something] $x; done" and I type those without thinking when I'm working in the shell. I haven't written any perl in probably 12 years, but I still use it for file manipulation.

deleted 3y ago

[deleted]

mpalmer 3y ago

jq has string interpolation too, so it can even get a little bit nicer: "<option value=\"/examples/\(.)/\">\(.)</option>"

mdaniel 3y ago
Yeah, but muscle memory bites me all the time and I put the backslash on the closing paren, too, because I'm so used to the regex usage of that syntax which needs them to match I also want to draw the reader's attention to the magic of |@uri <https://github.com/jqlang/jq/blob/jq-1.7/docs/content/manual/manual.yml#L2097 https://github.com/jqlang/jq/blob/jq-1.7/docs/content/manual...> for a bunch of cases, but doubly so in TFA's case where they're plugging strings into a URI context. Simple string concat often works great for "hello world", but the world is not always just hello, so one quick use of the filter and jq's got your back echo "the world's scary" | jq -Rr '"<option value=\"/examples/\(.|@uri)\">\(.)</value>"'

sam0x17 3y ago

> Back in the Bootstrap 3 days, there used to be a customizer. There is still a quite good bootstrap customizer at bootstrap.build btw ;)

rnemeth91 3y ago

If you'd like to use jq in any JetBrains IDE (e.g. IntelliJ), here's a plugin for it: https://plugins.jetbrains.com/plugin/23360-jqexpress https://plugins.jetbrains.com/plugin/23360-jqexpress