4 ms·
Wrote a tweet-fitting one-liner that downloads every csv-like file from every government open data portal that's public on Socrata. Gave me a fun reason to play
by chaps 4y ago
Wrote a tweet-fitting one-liner that downloads every csv-like file from every government open data portal that's public on Socrata. Gave me a fun reason to play with FIFO files to do makeshift recursion.
mkfifo f;echo \0>f&xargs -E, -a<(cat f) -I{} bash -c "wget http://api.us.socrata.com/api/catalog/v1?scroll_id={} -O-|jq -r '.results[]|.resource.id+"\","\"+.metadata.domain+"\"/api/views/\""+.resource.id+"\"/rows.csv\""'|tee <(cut -d, -f2|xargs wget -P'{}' --content-disposition)|sed q"|tee f