6 ms·

Jqfmt like gofmt, but for jq

159 points by Bluestein 1y ago

Hendrikto 1y ago
> Side note: Ever tried Googling for "jq formatter"? Reading search results is a nightmare since jq itself is, among other things, a formatter. That’s what I thought too, when I read the title. To clarify: This tool formats jq commands, not JSON itself.
- layer8 1y ago
  While it doesn’t help much for search in this case, the more specific term is “pretty-printer”.
- vanschelven 1y ago
  Which makes sense because jq, with no options, acts as a formatter by default. (it's about 50% of my jq usage).
xmonkee 1y ago
God I really abhor jq and it seems it's becoming a standard. I dislike it cause I'm too dumb to correctly dredge up it's incantations, and once a year I have to go reading their arcane docs. I suppose it's another fertile ground for LLM use.
- mdaniel 1y ago
  The bad news is that much like how "I'm just going to DSL this ..." inevitably morphs into a full-blown programming language[1], so too is the ubiquitous "gah, your language is too complex, I'm going to just use this other tool that implements my favorite 10% of the cases" which is a long way of saying: or else what? There's 100% no way that I'm going to ever, ever use <<python3 -c "import json, sys; print(json.load(sys.stdin)[...ohgawd...]">> and if you are, then more power to ya and jq apparently doesn't solve a problem you have 1: https://www.laws-of-software.com/laws/zawinski/ https://www.laws-of-software.com/laws/zawinski/
- ashwinsundar 1y ago
  A standard for what? It just makes JSON look nicer and more query-able. You don't have to use it.
  - xmonkee 1y ago
    A standard as in there is a cottage industry of tools and websites built around it now, like this one.
    - lxgr 1y ago
      Given the choice between a hypothetical standard that nobody wrote (or implemented) and a tool that organically grew complex enough to benefit from a standard, I'd rather have the latter. Users (i.e. not implementors) usually also don't read the standard – they read the docs (ideally containing lots of examples on top of a dry enumeration of options), or today indeed ask an LLM.
- pxc 1y ago
  What would "non-arcane" jq docs look like? I'm kind of in the same boat, being an infrequent jq user, but I've generally found the docs pretty easy to navigate.
- benreesman 1y ago
  It's a pretty good on/off-ramp into better tools. Going from arbitrary slop to something that's a reasonable input to `nixlang` or Dhall is pure win IMHO. I get a lot of use out of `jq` even though I prefer sounder systems than JSON.
- quotemstr 1y ago
  Hey. Don't hate on jq too much. It's a backdoor way to get functional programming past people's mental perceived complexity forcefields.
- porridgeraisin 1y ago
  Look at gron. Or fastgron, same thing but faster. It basically dumps the flattened js equation form of the json. You can then sed grep that whatever. Then you can gron --ungron and it will make it JSON again. It works beautifully, check out the examples in their docs. fx.wtf is also very nice as an interactive viewer with vim movements. Then jo for creating json. I found it much easier to use these three + core utils as compared to learning jq, lol.
- JimDabell 1y ago
  JMESPath is an alternative and has better tool support in some places (e.g. the AWS CLI).
guerrilla 1y ago
I thought this title was rot13 at first. :D
- Bluestein 1y ago
  Gubhtug V jnf gur bayl bar :) PS. Honestly, it's pretty close.-
kiitos 1y ago
Instead of making users enable every formatting rule explicitly e.g. jqfmt -ob -ar -op pipe It would be better if the tool enabled a common set of rules by default, so that `echo ... | jqfmt` actually did something useful :)
noperator 1y ago
Hey, author here. see also, sol: a de-minifier (formatter, exploder, beautifier) for shell one-liners https://github.com/noperator/sol https://github.com/noperator/sol I actually wrote jqfmt because I needed it for sol :)
- Bluestein 1y ago
  Your explanation of the *many* "meanings" of "sol" is gold :) PS. Happily, featured here: - https://news.ycombinator.com/item?id=41556088 https://news.ycombinator.com/item?id=41556088
s17n 1y ago
If you need to format your one-liner, maybe it shouldn't be a one liner? Anyway whether or not this tool is advisable its definitely cool, nice work!
- Bluestein 1y ago
  > If you need to format your one-liner, maybe it shouldn't be a one liner? Entirely correct, this point.- PS. May I also appreciate your comment, as far as form? You made both, valid, points.-
- noperator 1y ago
  My prototype one-liners usually turn into Go programs :)
  - Bluestein 1y ago
    Sic semper :)
mikeocool 1y ago
Been using fx (fx.wtf) as alternative to jq recently. Give you a nice javascript interface to do similar types of processing to what I would do with jq.
- noperator 1y ago
  if you like fx, then you'll love https://jless.io/ https://jless.io/
quotemstr 1y ago
jq is convenient, but I don't see the draw in building data processing pipelines on it. It's like writing complex software in shell. Recently, I found myself wanting to do a join by filename on two sets of about 300,000 files. Tried bashing my head against jq with INDEX and various tricks and couldn't get the runtime below minutes. Then I just gave up, fired up Python, loaded the dataset into Pandas, and did a join. Completed too fast to notice.
- porridgeraisin 1y ago
  > bash Did the `join` command help?
  - quotemstr 1y ago
    No. join in jq is just syntactic sugar over object property lookup anyway. Turns out index() was the expensive step.
    - porridgeraisin 1y ago
      I meant the coreutils join command.
- ghProTip 1y ago
  Created this tiny JavaScript project and published on npm for when you just can't be arsed remembering/learning JQ: https://www.npmjs.com/package/jsling https://www.npmjs.com/package/jsling . Not shilling it so much as saying.. IMHO, for a lot of use cases, there are far better approaches than dealing with the occasional JQ struggle bus ride.
ForOldHack 1y ago
Stop naming products after falling silverware. jq is sed for json data. gofmt is a GO source formatter jqfmt is like gofmt a go source formatter, but for json. So jqfmt is really json beautifier... Anyone with an ASR-33 for sale? rq?
- 0points 1y ago
  "fmt" is a reasonable generic abbreviation of "format".
yyyk 1y ago
All the examples in the comments start reminding me of XSLT. One day, this language may have the same reputation.
rwiggins 1y ago
Oh, fantastic. jq has become an integral part of work for me. I'll use this opportunity to plug the one-liner I use all the time, which summarizes the "structure" of a doc in a jq-able way: https://github.com/stedolan/jq/issues/243#issuecomment-48470943 https://github.com/stedolan/jq/issues/243#issuecomment-48470... (I didn't write it, I'm just a happy user) For example: $ curl -s 'https://ip-ranges.amazonaws.com/ip-ranges.json' | jq -r '[path(..)|map(if type=="number" then "[]" else tostring end)|join(".")|split(".[]")|join("[]")]|unique|map("."+.)|.[]' . .createDate .ipv6_prefixes .ipv6_prefixes[] .ipv6_prefixes[].ipv6_prefix .ipv6_prefixes[].network_border_group .ipv6_prefixes[].region .ipv6_prefixes[].service .prefixes .prefixes[] .prefixes[].ip_prefix .prefixes[].network_border_group .prefixes[].region .prefixes[].service .syncToken (except I have it aliased to "jq-structure" locally of course. also, if there's a new fancy way to do this, I'm all ears; I've been using this alias for like... almost a decade now :/) In the spirit of trying out jqfmt, let's see how it formats that one-liner... ~ echo '[path(..)|map(if type=="number" then "[]" else tostring end)|join(".")|split(".[]")|join("[]")]|unique|map("."+.)|.[]' | ~/go/bin/jqfmt -ob -ar -op pipe [ path(..) | map(if type == "number" then "[]" else tostring end) | join(".") | split(".[]") | join("[]") ] | unique | map("." + .) | .[]% ~ Not bad! Shame that jqfmt doesn't output a newline at the end, though. The errant `%` is zsh's partial line marker. Also, `-ob -ar -op pipe` seems like a pretty good set of defaults to me - I would prefer that over it (seemingly?) not doing anything with no flags. (At least for this sample snippet.)
- petercooper 1y ago
  Not anywhere near as sophisticated as yours but I have something vaguely similar for simplifying JSON documents (while maintaining what the data also looks like) for feeding to LLMs to help them code against: jq 'walk(if type == "array" then (if length > 0 then [.[0]] else . end) else . end)' So that 70,000+ line Amazon example of yours would boil down to: { "syncToken": "1753114994", "createDate": "2025-07-21-16-23-14", "prefixes": [ { "ip_prefix": "3.4.12.4/32", "region": "eu-west-1", "service": "AMAZON", "network_border_group": "eu-west-1" } ], "ipv6_prefixes": [ { "ipv6_prefix": "2600:1f69:7400::/40", "region": "mx-central-1", "service": "AMAZON", "network_border_group": "mx-central-1" } ] } .. which is easier/cheaper to feed to an LLM for getting it to write code to process, etc. than the multi-megabyte original.

159 points by Bluestein 1y ago

> Side note: Ever tried Googling for "jq formatter"? Reading search results is a nightmare since jq itself is, among other things, a formatter. That’s what I thought too, when I read the title. To clarify: This tool formats jq commands, not JSON itself.

layer8 1y ago
While it doesn’t help much for search in this case, the more specific term is “pretty-printer”.
vanschelven 1y ago
Which makes sense because jq, with no options, acts as a formatter by default. (it's about 50% of my jq usage).

xmonkee 1y ago

God I really abhor jq and it seems it's becoming a standard. I dislike it cause I'm too dumb to correctly dredge up it's incantations, and once a year I have to go reading their arcane docs. I suppose it's another fertile ground for LLM use.

mdaniel 1y ago
The bad news is that much like how "I'm just going to DSL this ..." inevitably morphs into a full-blown programming language[1], so too is the ubiquitous "gah, your language is too complex, I'm going to just use this other tool that implements my favorite 10% of the cases" which is a long way of saying: or else what? There's 100% no way that I'm going to ever, ever use <<python3 -c "import json, sys; print(json.load(sys.stdin)[...ohgawd...]">> and if you are, then more power to ya and jq apparently doesn't solve a problem you have 1: https://www.laws-of-software.com/laws/zawinski/ https://www.laws-of-software.com/laws/zawinski/
ashwinsundar 1y ago
A standard for what? It just makes JSON look nicer and more query-able. You don't have to use it.
- xmonkee 1y ago
  A standard as in there is a cottage industry of tools and websites built around it now, like this one.
  - lxgr 1y ago
    Given the choice between a hypothetical standard that nobody wrote (or implemented) and a tool that organically grew complex enough to benefit from a standard, I'd rather have the latter. Users (i.e. not implementors) usually also don't read the standard – they read the docs (ideally containing lots of examples on top of a dry enumeration of options), or today indeed ask an LLM.
pxc 1y ago
What would "non-arcane" jq docs look like? I'm kind of in the same boat, being an infrequent jq user, but I've generally found the docs pretty easy to navigate.
benreesman 1y ago
It's a pretty good on/off-ramp into better tools. Going from arbitrary slop to something that's a reasonable input to `nixlang` or Dhall is pure win IMHO. I get a lot of use out of `jq` even though I prefer sounder systems than JSON.
quotemstr 1y ago
Hey. Don't hate on jq too much. It's a backdoor way to get functional programming past people's mental perceived complexity forcefields.
porridgeraisin 1y ago
Look at gron. Or fastgron, same thing but faster. It basically dumps the flattened js equation form of the json. You can then sed grep that whatever. Then you can gron --ungron and it will make it JSON again. It works beautifully, check out the examples in their docs. fx.wtf is also very nice as an interactive viewer with vim movements. Then jo for creating json. I found it much easier to use these three + core utils as compared to learning jq, lol.
JimDabell 1y ago
JMESPath is an alternative and has better tool support in some places (e.g. the AWS CLI).

guerrilla 1y ago

I thought this title was rot13 at first. :D

Bluestein 1y ago
Gubhtug V jnf gur bayl bar :) PS. Honestly, it's pretty close.-

kiitos 1y ago

Instead of making users enable every formatting rule explicitly e.g. jqfmt -ob -ar -op pipe It would be better if the tool enabled a common set of rules by default, so that `echo ... | jqfmt` actually did something useful :)

noperator 1y ago

Hey, author here. see also, sol: a de-minifier (formatter, exploder, beautifier) for shell one-liners https://github.com/noperator/sol https://github.com/noperator/sol I actually wrote jqfmt because I needed it for sol :)

Bluestein 1y ago
Your explanation of the *many* "meanings" of "sol" is gold :) PS. Happily, featured here: - https://news.ycombinator.com/item?id=41556088 https://news.ycombinator.com/item?id=41556088

s17n 1y ago

If you need to format your one-liner, maybe it shouldn't be a one liner? Anyway whether or not this tool is advisable its definitely cool, nice work!

Bluestein 1y ago
> If you need to format your one-liner, maybe it shouldn't be a one liner? Entirely correct, this point.- PS. May I also appreciate your comment, as far as form? You made both, valid, points.-
noperator 1y ago
My prototype one-liners usually turn into Go programs :)
- Bluestein 1y ago
  Sic semper :)

mikeocool 1y ago

Been using fx (fx.wtf) as alternative to jq recently. Give you a nice javascript interface to do similar types of processing to what I would do with jq.

noperator 1y ago
if you like fx, then you'll love https://jless.io/ https://jless.io/

quotemstr 1y ago

jq is convenient, but I don't see the draw in building data processing pipelines on it. It's like writing complex software in shell. Recently, I found myself wanting to do a join by filename on two sets of about 300,000 files. Tried bashing my head against jq with INDEX and various tricks and couldn't get the runtime below minutes. Then I just gave up, fired up Python, loaded the dataset into Pandas, and did a join. Completed too fast to notice.

porridgeraisin 1y ago
> bash Did the `join` command help?
- quotemstr 1y ago
  No. join in jq is just syntactic sugar over object property lookup anyway. Turns out index() was the expensive step.
  - porridgeraisin 1y ago
    I meant the coreutils join command.
ghProTip 1y ago
Created this tiny JavaScript project and published on npm for when you just can't be arsed remembering/learning JQ: https://www.npmjs.com/package/jsling https://www.npmjs.com/package/jsling . Not shilling it so much as saying.. IMHO, for a lot of use cases, there are far better approaches than dealing with the occasional JQ struggle bus ride.

ForOldHack 1y ago

Stop naming products after falling silverware. jq is sed for json data. gofmt is a GO source formatter jqfmt is like gofmt a go source formatter, but for json. So jqfmt is really json beautifier... Anyone with an ASR-33 for sale? rq?

0points 1y ago
"fmt" is a reasonable generic abbreviation of "format".

yyyk 1y ago

All the examples in the comments start reminding me of XSLT. One day, this language may have the same reputation.

rwiggins 1y ago

Oh, fantastic. jq has become an integral part of work for me. I'll use this opportunity to plug the one-liner I use all the time, which summarizes the "structure" of a doc in a jq-able way: https://github.com/stedolan/jq/issues/243#issuecomment-48470943 https://github.com/stedolan/jq/issues/243#issuecomment-48470... (I didn't write it, I'm just a happy user) For example: $ curl -s 'https://ip-ranges.amazonaws.com/ip-ranges.json' | jq -r '[path(..)|map(if type=="number" then "[]" else tostring end)|join(".")|split(".[]")|join("[]")]|unique|map("."+.)|.[]' . .createDate .ipv6_prefixes .ipv6_prefixes[] .ipv6_prefixes[].ipv6_prefix .ipv6_prefixes[].network_border_group .ipv6_prefixes[].region .ipv6_prefixes[].service .prefixes .prefixes[] .prefixes[].ip_prefix .prefixes[].network_border_group .prefixes[].region .prefixes[].service .syncToken (except I have it aliased to "jq-structure" locally of course. also, if there's a new fancy way to do this, I'm all ears; I've been using this alias for like... almost a decade now :/) In the spirit of trying out jqfmt, let's see how it formats that one-liner... ~ echo '[path(..)|map(if type=="number" then "[]" else tostring end)|join(".")|split(".[]")|join("[]")]|unique|map("."+.)|.[]' | ~/go/bin/jqfmt -ob -ar -op pipe [ path(..) | map(if type == "number" then "[]" else tostring end) | join(".") | split(".[]") | join("[]") ] | unique | map("." + .) | .[]% ~ Not bad! Shame that jqfmt doesn't output a newline at the end, though. The errant `%` is zsh's partial line marker. Also, `-ob -ar -op pipe` seems like a pretty good set of defaults to me - I would prefer that over it (seemingly?) not doing anything with no flags. (At least for this sample snippet.)

petercooper 1y ago
Not anywhere near as sophisticated as yours but I have something vaguely similar for simplifying JSON documents (while maintaining what the data also looks like) for feeding to LLMs to help them code against: jq 'walk(if type == "array" then (if length > 0 then [.[0]] else . end) else . end)' So that 70,000+ line Amazon example of yours would boil down to: { "syncToken": "1753114994", "createDate": "2025-07-21-16-23-14", "prefixes": [ { "ip_prefix": "3.4.12.4/32", "region": "eu-west-1", "service": "AMAZON", "network_border_group": "eu-west-1" } ], "ipv6_prefixes": [ { "ipv6_prefix": "2600:1f69:7400::/40", "region": "mx-central-1", "service": "AMAZON", "network_border_group": "mx-central-1" } ] } .. which is easier/cheaper to feed to an LLM for getting it to write code to process, etc. than the multi-megabyte original.