4 ms·
Here here, I agree. In the most recent versions we have seen speed increases. The fact that polars exist shows that there is a tone of low hanging fruit. There
by RandomWorker 4y ago
Here here, I agree. In the most recent versions we have seen speed increases. The fact that polars exist shows that there is a tone of low hanging fruit.
There is also dask to increase parallelism and performance which I’ve used on some massive datasets 200GB+
- __mharrison__ 4y agoPandas (the API) is also getting better at big data. I'm an advisor at a company, Ponder, that will take your Pandas code and execute it on "big data".
- tccole 4y agoAren’t there a bunch of plugins for that kind of thing?
- maegul 4y agoIsn't the low hanging fruit that polars picked: "how about lazy evaluation to allow the query to be optimised?" ... which is mostly anathema to the design of pandas?