5 ms·
Given these examples, I don't really see why anyone would replace Pandas with Polars for existing workloads. Memory's not an issue because they already bought t
by ericpauley 4y ago
Given these examples, I don't really see why anyone would replace Pandas with Polars for existing workloads. Memory's not an issue because they already bought the DIMMs (which have gotten insanely cheap). Personally I don't find the speed of Pandas to be that bad if you use it smartly (e.g., not parsing data more often than you have to).
- jerpint 4y agoI agree nothing in that article was a compelling argument
- minimaxir 4y agoWorking with Actual Big Data (tm) will inevitably hit issues with both aspects in pandas, where polars can theoretically help. Granted, the real correct solution for manipulating Actual Big Data (tm) is at the data warehouse level itself before it even hits Python but that's a topic for another day.
- __mharrison__ 4y agoPonder is working on making Pandas actually work with really big data. (Disclaimer: I'm an advisor for them.)
- __mharrison__ 4y agoThis article suffers from the same thing that most Pandas articles or books suffer from. Showing operations in isolation. I rarely do one thing to a dataset. Show me what real world code looks like. Pandas and Polars are both awesome. Pandas is more entrenched and is a solid choice these days (and getting better for big data). Polars is an awesome up and coming library. Competition is good for the Python Data ecosystem. (Disclaimer: I've authored multiple Pandas books and have taught courses on both Pandas and Polars.)