5 ms·
pandas + fastparquet fairly lightweight. but yes I would love to see a simple c++/golang binary that's just a simple csv2parq call.
by meritt 6y ago
pandas + fastparquet fairly lightweight. but yes I would love to see a simple c++/golang binary that's just a simple csv2parq call.
- MrPowers 6y agoNewer versions of Pandas don't even need fastparquet anymore. This code works: import pandas as pd df = pd.read_csv('data/us_presidents.csv') df.to_parquet('tmp/us_presidents.parquet')
- meritt 6y agoNice! Does that work alongside reading in via chunks and writing via row_groups? If I have a 500GB CSV will it work?