4 ms·
I have this shell function defined: csv_to_json () { python -c 'import csv, json, sys; print(json.dumps([dict(r) for r in csv.DictReader(sys.stdin)]))'
by chrisshroba 3y ago
I have this shell function defined:
csv_to_json () {
python -c 'import csv, json, sys; print(json.dumps([dict(r) for r in csv.DictReader(sys.stdin)]))' | jq .
}
It converts a csv to a json list of objects, mapping column names to values. I find it way easier to then operate on json by filtering with jq or gron, or just pasting it into other tools for post-processing. The jq at the end isn't necessary but makes for nice formatting!
- vladxyz 3y agoI'm two weeks late, but you could do something like python -c 'import csv, json, sys; [print(json.dumps(r)) for r in csv.DictReader(sys.stdin)]' | jq -s . to stream the csv rows instead of reading the whole file into ram.
- sgarland 3y agoThat won’t scale well to files approaching or exceeding system RAM, though. Also, unless it’s the coloring you’re concerned with, you could replace the pipe to jq with pprint, a native Python library.
- chrisshroba 3y agopprint won't work here because the output is a json string, not a python object, so pprint'ing it will just print the string as is. And plus, pprint doesn't output valid json (it uses single quotes, for example), so the output isn't usable for piping into other filters and programs. And it's true that it won't scale well but it still has worked every time I've wanted it to :)