3 ms·
Parquet also encodes the physical layout using footers [1], as does ORC [2]. Perhaps the author meant support for semi-structured data, like the spans you menti
by sa46 2y ago
Parquet also encodes the physical layout using footers [1], as does ORC [2]. Perhaps the author meant support for semi-structured data, like the spans you mention.
[1]: https://parquet.apache.org/docs/file-format/ https://parquet.apache.org/docs/file-format/
[2]: https://orc.apache.org/specification/ORCv2/#file-tail https://orc.apache.org/specification/ORCv2/#file-tail
- danking00 2y agoYeah we should be more clear in our description about how our footers differ from Parquet. Parquet is a bit more prescriptive; for example, it requires row groups which are not required by Vortex. If you have a column with huge values and another column of 8 bit ints, they can be paged separately, if you like.