3 ms·
I too can never remember jq syntax when I need to. I usually just end up writing a Python script to extract the part of the JSON I need.
by umvi 3y ago
I too can never remember jq syntax when I need to. I usually just end up writing a Python script to extract the part of the JSON I need.
- paiute 3y agoI would say the same thing about sed and awk. I have a few basics down solid, but anything complex is best done with python.
- defrost 3y agoIf JQ is too much, see GRON &| Miller gron transforms JSON into discrete assignments to make it easier to grep for what you want https://github.com/tomnomnom/gron https://github.com/tomnomnom/gron Miller is like awk, sed, cut, join, and sort for data formats such as CSV, TSV, JSON, JSON https://github.com/johnkerl/miller https://github.com/johnkerl/miller
- e40 3y agoGron looks interesting, but I wish there was an option for bash output!
- kitd 3y agoThe main benefit (indeed the purpose) of gron is that the output is much easier to manipulate into what you need using basic gnu utils, instead of trying to shoehorn it into jq's syntax.
- btschaegg 3y agoHaving used gron in bash scripts, I think "…is much easier to manipulate […] using gnu utils" is overselling it somewhat. You can `grep` through it, yes. But since the output is still structured to be Javascript (which makes it nice in some more immediate ways if you're working interactively) makes it tricky to deal with with e.g. `awk` -- you'd still have to parse the JS line and implement string unescaping, which still makes it more complicated than necessary. And if you work e.g. on JSON in a bash variable, running `gron -v` also is cumbersome enough for me to want to use another tool. So, I still like `gron`, I just think it has a rather small niche. One that maybe could be a bit larger if there was a way for it to output delimited records with user-specified delimiters (provided you know the output enough to be sure they don't appear in any strings).
- kitd 3y agoNushell is also good for this, especially when you need shell capabilities too. https://www.nushell.sh/ https://www.nushell.sh/
- eternityforest 3y agoReally cool but... Python exists. If I am gonna do anything with JSON that is probably well into "Time to use a full programming language" territory
- defrost 3y agoFWiW my preference for JSON related data transforms and driven computation is almost always JQ - it is a Turing-complete interpreter language with JSON iterator as the first-citizen object. I suggested the above for those that can't justify the time it takes to make JQ perform well for whatever relatively minor needs they have. JQ is back under active development and there are a number of optimised | Go | Rust spinoff github projects that'll likely do some interesting work now that the mainline is starting to release again.
- raincole 3y agoThere is nothing wrong with choosing a language like Python/Ruby over "command line sorcery".
- ElCapitanMarkla 3y agoChatGPT is a fantastic usecase for this. Last week I had to extract a bunch of data that was in some nested arrays, pasted a json sample into ChatGPT and asked it to use jq to extract certain fields when certain conditions were met, a couple of tweaks later and I had exactly what I needed.
- Grimburger 3y agoI tend to frustrate colleagues with this but it's honestly so much more manageable. A python script is the sane extensible choice rather than some esoteric bash incantation that you have no clue about 6 months after writing it.
- elesiuta 3y ago> I too can never remember jq syntax when I need to. I usually just end up writing a Python script Same here! That's why for small things I made pyxargs [1] to use python in the shell. In another thread I also just learned of pyp [2] which I haven't tried yet but looks like it's even better for this use case. [1] https://github.com/elesiuta/pyxargs https://github.com/elesiuta/pyxargs [2] https://github.com/hauntsaninja/pyp https://github.com/hauntsaninja/pyp
- fragmede 3y agoHonestly, whenever I need to use jq, I just search my bash history (fzf ftw) for the last time I used it. How'd I get it to work the first time? lost to the sands of time...