3 ms·
Naive pandas is going to limit you to RAM sizes when parquet is deliberately meant for huge datasets.
by 0cf8612b2e1e 4y ago
Naive pandas is going to limit you to RAM sizes when parquet is deliberately meant for huge datasets.
- FridgeSeal 4y agoAnyone reading who is currently using pandas and running into this issue-polars is generally faster, and now supports (still new though) out-of-core workloads.