3 ms·
Right, there's all sorts of metadata and often stats included in any parquet file: https://github.com/apache/parquet-format#file-format https://github.com/apach
by choppaface 3y ago
Right, there's all sorts of metadata and often stats included in any parquet file: https://github.com/apache/parquet-format#file-format https://github.com/apache/parquet-format#file-format
The offsets of said metadata are well-defined (i.e. in the footer) so for S3 / blob storage so long as you can efficiently request a range of bytes you can pull the metadata without having to read all the data.