4 ms·
Pandas often has significant memory overhead, so it's not uncommon to need ~3-5x the amount of memory as your file size. Polars and DuckDB are much better abou
by datadrivenangel 2y ago
Pandas often has significant memory overhead, so it's not uncommon to need ~3-5x the amount of memory as your file size.
Polars and DuckDB are much better about memory management.
- marton78 2y agoAlso, Polars has a sane and well thought out API, unlike the hot streaming mess that is Pandas.