3 ms·
On the other hand because it is column storage if you want to do the sum of one column grouped by another column, and there is a 100 other columns, it is way fa
by cm2187 4y ago
On the other hand because it is column storage if you want to do the sum of one column grouped by another column, and there is a 100 other columns, it is way faster than csv since you only read the parts of the file you need.
- crabbone 4y agoThis is an oversimplification. You will have to read the index to figure out where the parts you need to read are. And if your columns don't align well with the blocks / pages of the storage you are using, this "optimization" may turn out to be worthless, or, perhaps even harmful, if your system is starving for memory.