4 ms·
I was recently working with a 38GB compressed text dataset locally. I can't remember how big it was in parquet. It's literally one line of code. See http://lab
by nl 10y ago
I was recently working with a 38GB compressed text dataset locally. I can't remember how big it was in parquet.
It's literally one line of code. See http://labs.vistarmedia.com/2016/12/27/indexing-json-logs-with-parquet.html http://labs.vistarmedia.com/2016/12/27/indexing-json-logs-wi... for an example (except you write to a local file system rather than HDFS).
- eximius 10y agoThat looks very helpful, I'll have to check it out!