3 ms·
I dont get the either/or posts here about polars. Some times I do duckdb sql queries against a polars df, some times I convert a duckdb result to polars. Its gr
by fastasucan 9mo ago
I dont get the either/or posts here about polars. Some times I do duckdb sql queries against a polars df, some times I convert a duckdb result to polars. Its great to use whatever is best for the situation.
- RobinL 9mo agoI think that's totally fine for individual work, but in larger data engineering teams it's less good to switch between tools because other people may have to maintain your code. That said, polars is good, and if the team agree to standardise on it then that's a totally reasonable choice. I guess one of my reservations is I've been (historically) burned by decisions within data eng teams to use pandas, causing all sorts of problems with data typing and memory and eventually having to rewrite it all. But I accept polars doesn't suffer from the same problems (and actually some of them are even mitigated in more recent versions of pandas)