3 ms·
This is time partitioning, not indexing, but it's commonly done in data lakes for raw telemetry/log data. mydata/ 2019/ 11/
by kyllo 7y ago
This is time partitioning, not indexing, but it's commonly done in data lakes for raw telemetry/log data.
mydata/
2019/
11/
12/
2019111200.csv
2019111201.csv
Where each one of those CSV files could be 10-100 GB in size.
Usually you want to process it into a columnar format like Parquet from there, though