I have a somewhat related question: What is the current best practice way to parallelize reading many parquet files on S3 using pyarrow? Deep down, it's just l

I have a somewhat related question: What is the current best practice way to parallelize reading many parquet files on S3 using pyarrow? Deep down, it's just lots of HTTP requests.