4 ms·

I hate pandas. Polars is significantly better designed. Please, everyone, stop using Pandas.

by nathan_compton 3y ago

freilanzer 3y ago
Polars seems somewhat cumbersome to use.
- haswell 3y ago
  I’m a fan of pandas, but it is also somewhat cumbersome to use until one has learned its peculiarities.
  - nathan_compton 3y ago
    Pandas has a variety of design elements I would characterize as "insane."
    - freilanzer 3y ago
      Could you give examples? I don't use a lot of Pandas.
boringg 3y ago
Espouse the benefits of polars vs pandas and ill consider it. At this point i know pandas very well and, while it had some nuances, its pretty good at what it does!
- seanhunter 3y ago
  Not the OP but I’ll give my opinion. Ultimately they are similar but polars has definitely benefited from “going second” ie seeing a successful example and how people use it and knocking off some rough edges. 1. Polars has a narrower API that seems a bit cleaner and more logical vs pandas is more maximalist. Matter of taste but https://franzdiebold.github.io/polars-cheat-sheet/Polars_cheat_sheet.pdf https://franzdiebold.github.io/polars-cheat-sheet/Polars_che... Will give you a flavour 2. Polars perf seems quite a lot better. This may not matter for your use case but if it does it does. 3. Polars supports both eager and lazy evaluation modes and you can choose whichever makes sense for your use case. I like this but sone people find lazy eval confusing to debug. 4. polars can do things like parallelization under the hood 5. Polars is less inclined to guess what you mean about things like type conversions and will instead fail and throw until you fix your code to be more explicit. Pandas is a bit more lassez faire. I guess this reflects the dev philosophy of the respective teams. Here’s someone else’s writeup with some benchmarks https://towardsdatascience.com/pandas-vs-polars-a-syntax-and-speed-comparison-5aa54e27497e https://towardsdatascience.com/pandas-vs-polars-a-syntax-and...
  - boringg 3y ago
    Thanks interesting diffs. I think functionality is the missing diff at this point. Appreciate the parallelization.
czbond 3y ago
Any experience with Polars & ArticDB? (I've been wondering how Polars could improve how I use Artic.... 20k+ smaller dataframes updated often from squirts back from IoT devices)
elforce002 3y ago
I don't hate pandas but after using R and Dplyr, I didn't want to touch pandas ever again. Now with Polars, I'm enjoying EDAs and replacing pandas on our ML stack has been fun. I could see a case being made for Pandas if they emulate Dplyr.

I hate pandas. Polars is significantly better designed. Please, everyone, stop using Pandas.

freilanzer 3y ago

Polars seems somewhat cumbersome to use.

haswell 3y ago
I’m a fan of pandas, but it is also somewhat cumbersome to use until one has learned its peculiarities.
- nathan_compton 3y ago
  Pandas has a variety of design elements I would characterize as "insane."
  - freilanzer 3y ago
    Could you give examples? I don't use a lot of Pandas.

boringg 3y ago

Espouse the benefits of polars vs pandas and ill consider it. At this point i know pandas very well and, while it had some nuances, its pretty good at what it does!

seanhunter 3y ago
Not the OP but I’ll give my opinion. Ultimately they are similar but polars has definitely benefited from “going second” ie seeing a successful example and how people use it and knocking off some rough edges. 1. Polars has a narrower API that seems a bit cleaner and more logical vs pandas is more maximalist. Matter of taste but https://franzdiebold.github.io/polars-cheat-sheet/Polars_cheat_sheet.pdf https://franzdiebold.github.io/polars-cheat-sheet/Polars_che... Will give you a flavour 2. Polars perf seems quite a lot better. This may not matter for your use case but if it does it does. 3. Polars supports both eager and lazy evaluation modes and you can choose whichever makes sense for your use case. I like this but sone people find lazy eval confusing to debug. 4. polars can do things like parallelization under the hood 5. Polars is less inclined to guess what you mean about things like type conversions and will instead fail and throw until you fix your code to be more explicit. Pandas is a bit more lassez faire. I guess this reflects the dev philosophy of the respective teams. Here’s someone else’s writeup with some benchmarks https://towardsdatascience.com/pandas-vs-polars-a-syntax-and-speed-comparison-5aa54e27497e https://towardsdatascience.com/pandas-vs-polars-a-syntax-and...
- boringg 3y ago
  Thanks interesting diffs. I think functionality is the missing diff at this point. Appreciate the parallelization.

czbond 3y ago

Any experience with Polars & ArticDB? (I've been wondering how Polars could improve how I use Artic.... 20k+ smaller dataframes updated often from squirts back from IoT devices)

elforce002 3y ago

I don't hate pandas but after using R and Dplyr, I didn't want to touch pandas ever again. Now with Polars, I'm enjoying EDAs and replacing pandas on our ML stack has been fun. I could see a case being made for Pandas if they emulate Dplyr.