4 ms·
The builtin CSV/TSV parsing seems the killer feature here, at least for me. I've been using SQL on CSV and JSON lately as it's mostly easiest for me, but I can
by michaelcampbell 3y ago
The builtin CSV/TSV parsing seems the killer feature here, at least for me.
I've been using SQL on CSV and JSON lately as it's mostly easiest for me, but I can think back a few times where I've used an *awk for a specific need, AND had to deal with CSV parsing at the same time.
- arp242 3y agoBoth GNU awk and "one true awk" added --csv last year, so as the README note mentions you probably don't need frawk for this (any more).
- tooltower 3y agoWhat's the common tool people use these days for manipulating json in an awk-like manner?
- hermitcrab 3y agoProbably: https://jqlang.github.io/jq/ https://jqlang.github.io/jq/
- rmetzler 3y agoI would say that lots of people use jq. But it’s not really awk-like. You could probably use gron with awk pretty easily, since gron stands for “greppable json”. But I also wonder how an awk with native json support would look like.
- mbivert 3y agoFood for thought: have a look at this paper[0] about structural regular expressions. The author (Rob Pike) sketches in the last section an awk-support. I remember using such regexps a while ago to tweak indented JSON and JSON-like data (the indentation allowed to loop on hashes easily). An awk with json support would for the most part need to be able to loop on hashes and arrays, and provide ways to travel in-depth. So far regular awk can travel through arrays (line-separator), and "in-depth" (e.g. nested "arrays") via regular loops & cie. Probably easier to think about it with a few concrete examples though. [0]: http://doc.cat-v.org/bell_labs/structural_regexps/se.pdf http://doc.cat-v.org/bell_labs/structural_regexps/se.pdf
- sceadu 3y agogron (https://github.com/tomnomnom/gron https://github.com/tomnomnom/gron) to transform it and query and then invert the transformation?
- tastyminerals2 3y agoIf you need just csv/tsv parsing, you can also take a look at https://github.com/eBay/tsv-utils https://github.com/eBay/tsv-utils
- fuzztester 3y agointerestingly, i had read somewhere earlier that those utilities were written in D. confirmed it by looking at the site just now.
- gpvos 3y agoI only found out recently that my favourite csv-munging tool, csvtool, is written in OCaml.
- tastyminerals2 3y agoYes, and with heavy use of one of the biggest D strengths -- CTFE (compile time eval). Great lib!
- fuzztester 3y agoHow do you use SQL on JSON? You mean only for strictly tabular JSON, i.e. no nesting beyond rows and columns?
- michaelcampbell 3y agoduckdb, as noted by someone else as well.
- kermatt 3y agoSpark and Drill have options, but DuckDB makes it almost trivial: https://duckdb.org/docs/data/json/overview https://duckdb.org/docs/data/json/overview
- fuzztester 3y agoThanks, will check that out.