3 ms·
Um, yes. From the article: "The simplest and most common way to implement indexing is by naming files in a directory:" mydata/ 2019-Jan.csv
by Animats 7y ago
Um, yes. From the article:
"The simplest and most common way to implement indexing is by naming files in a directory:"
mydata/
2019-Jan.csv
2019-Feb.csv
2019-Mar.csv
2019-Apr.csv
Um.
- kyllo 7y agoThis is time partitioning, not indexing, but it's commonly done in data lakes for raw telemetry/log data. mydata/ 2019/ 11/ 12/ 2019111200.csv 2019111201.csv Where each one of those CSV files could be 10-100 GB in size. Usually you want to process it into a columnar format like Parquet from there, though