4 ms·
Parquet is column oriented and so row-based manipulation can be inefficient
by jjtheblunt 2y ago
Parquet is column oriented and so row-based manipulation can be inefficient
- fock 2y ago> Hierarchically, a file consists of one or more row groups. https://parquet.apache.org/docs/concepts/ https://parquet.apache.org/docs/concepts/ Maybe the file in question only has one row group. Which would be weird, because the creator had to go out of their way to make it happen.
- jjtheblunt 2y agoYep. I use it all the time. But, as you said, depends on specific layouts, so can’t expect it to be row-convenient .