5 ms·

I'm the author of rq, and it's great to hear about your use-cases! Feel free to submit a feature request about the raw string use-case, I've been thinking abou

I'm the author of rq, and it's great to hear about your use-cases! Feel free to submit a feature request about the raw string use-case, I've been thinking about it for a while but some people have wanted me to add line support (maybe -l and -L?) and some people have wanted raw text support (maybe -r and -R?) while other people still have been talking about TSV/CSV support (not sure how that would look tbh...). I'm not sure what to do exactly, so feel free to tell me!

danso 10y ago
Aw dang, I wish I knew about this earlier. May have made teaching command-line data processing to beginners quite a bit easier. Example lesson using Spotify API (which is still public and free): http://www.compciv.org/recipes/data/touring-the-spotify-api/ http://www.compciv.org/recipes/data/touring-the-spotify-api/ (just noticed that you used the Spotify API too for the demo :) https://github.com/dflemstr/rq/blob/master/doc/demo.md https://github.com/dflemstr/rq/blob/master/doc/demo.md) I personally like using jq but not sure how easy-to-grok it is for people who are new to both the command-line and to serialized data structures in general. Will have give rq a spin (waiting for it to compile on my machine) but it looks quite nice, at least for my own uses, which heavily involve YAML. In terms of TSV/CSV support, you may want to checkout csvjson, which is part of the csvkit command-line suite: http://csvkit.readthedocs.io/en/0.9.1/scripts/csvjson.html http://csvkit.readthedocs.io/en/0.9.1/scripts/csvjson.html csvkit is my go-to tool for data, so much that I rarely ever use SQL for anything other than when I actually need a database. I suppose if rq were to be my go-to data-processing tool, I'd probably do this: csvjson mydata.csv | rq ... Unless I'm missing it in the documents, there doesn't appear to be a way to convert to CSV? That would be helpful. FWIW, csvkit has a tool named in2csv, which will read from line-delimited JSON: curl -s 'https://api.spotify.com/v1/search?q=rihanna&type=track&limit=50' \ | rq 'at "tracks.items" | spread | flatMap artists' \ | in2csv -f ndjson http://csvkit.readthedocs.io/en/stable/scripts/in2csv.html http://csvkit.readthedocs.io/en/stable/scripts/in2csv.html

danso 10y ago

Aw dang, I wish I knew about this earlier. May have made teaching command-line data processing to beginners quite a bit easier. Example lesson using Spotify API (which is still public and free): http://www.compciv.org/recipes/data/touring-the-spotify-api/ http://www.compciv.org/recipes/data/touring-the-spotify-api/ (just noticed that you used the Spotify API too for the demo :) https://github.com/dflemstr/rq/blob/master/doc/demo.md https://github.com/dflemstr/rq/blob/master/doc/demo.md) I personally like using jq but not sure how easy-to-grok it is for people who are new to both the command-line and to serialized data structures in general. Will have give rq a spin (waiting for it to compile on my machine) but it looks quite nice, at least for my own uses, which heavily involve YAML. In terms of TSV/CSV support, you may want to checkout csvjson, which is part of the csvkit command-line suite: http://csvkit.readthedocs.io/en/0.9.1/scripts/csvjson.html http://csvkit.readthedocs.io/en/0.9.1/scripts/csvjson.html csvkit is my go-to tool for data, so much that I rarely ever use SQL for anything other than when I actually need a database. I suppose if rq were to be my go-to data-processing tool, I'd probably do this: csvjson mydata.csv | rq ... Unless I'm missing it in the documents, there doesn't appear to be a way to convert to CSV? That would be helpful. FWIW, csvkit has a tool named in2csv, which will read from line-delimited JSON: curl -s 'https://api.spotify.com/v1/search?q=rihanna&type=track&limit=50' \ | rq 'at "tracks.items" | spread | flatMap artists' \ | in2csv -f ndjson http://csvkit.readthedocs.io/en/stable/scripts/in2csv.html http://csvkit.readthedocs.io/en/stable/scripts/in2csv.html