4 ms·
csv files are extremely easy to import in postgres, and 10 M rows (assuming not very large) isn't much to compute even in a 6 or 7 year old laptop. Keep it in m
by dorfsmay 8y ago
csv files are extremely easy to import in postgres, and 10 M rows (assuming not very large) isn't much to compute even in a 6 or 7 year old laptop. Keep it in mind if you've got something slightly more complicated to analyse.
- hobs 8y agoIf SQL is your game but you dont want to get PG setup - try SQLite - wget "https://data.cityofnewyork.us/api/views/kku6-nxdu/rows.csv" rows.csv sqlite3 .mode csv .import ./rows.csv newyorkdata SELECT * FROM newyorkdata ORDER BY `COUNT PARTICIPANTS`;
- mxuribe 8y agoNice! Talk about being a productive engineer! Kudos!
- jillesvangurp 8y agoOr use csvkit ...
- hobs 8y agoOh nice, I hadn't seen this before, so a similar query would be a bit shorter! wget "https://data.cityofnewyork.us/api/views/kku6-nxdu/rows.csv" csvsql --query "SELECT * FROM newyorkdata ORDER BY `COUNT PARTICIPANTS`" rows.csv > new.csv