Column-oriented formats such as Parquet can be awful too. For example, if you have five columns of numbers you want to multiply together, Parquet is going to be

Column-oriented formats such as Parquet can be awful too. For example, if you have five columns of numbers you want to multiply together, Parquet is going to be the worst option available because of the extreme cache thrashing that the CPU will encounter. Structure packing[1] and consideration of locality of reference[2] would need to be applied for high performance applications where a computer scientist has considered the algorithm needing to be implemented and the most efficient data format that the source data would need to be provided in. [1] http://www.catb.org/esr/structure-packing/ http://www.catb.org/esr/structure-packing/ [2] https://en.wikipedia.org/wiki/Locality_of_reference https://en.wikipedia.org/wiki/Locality_of_reference