Understand the format of your data. Look at the parquet file metadata: use whatever tool you want for that. The Python parquet library is useful and supports s

Understand the format of your data. Look at the parquet file metadata: use whatever tool you want for that. The Python parquet library is useful and supports s3. How big are your row groups? If it’s one large row group then you will run into this issue. What’s the number of rows in each row group?