5 ms·

I like that there is an alternative to Pandas, but unfortunately it uses the same dataframes paradigm. I know I'm in the minority, but I never liked dataframes.

I like that there is an alternative to Pandas, but unfortunately it uses the same dataframes paradigm. I know I'm in the minority, but I never liked dataframes. They are too "R"-like, don't fit into Python, and try to do too many things at once. Columns have names, except when they don't. Things (rows? individual cells?) have types which is sometimes enforced and sometimes not. If you don't follow a recipe exactly, your dataframe will be subtly different and might not work. A lot of code mixes mutation in place and creating new dataframes, and so on. And Pandas lends itself to a special chained method syntax which is almost it's own DSL in Python.

ritchie46 4y ago
> Columns have names, except when they don't. Things (rows? individual cells?) They are not a fit for every problem. Please don't try to use them for that. For normalized tabular data, it is a good fit. > have types which is sometimes enforced and sometimes not. If you don't follow a recipe exactly, your dataframe will be subtly different and might not work. Polars is really strict on types. A type is always determined by the input datatype and the operation applied. You can check the schema before you run the query. Polars will produce the same output datatype as defined by its logical plan. > A lot of code mixes mutation in place and creating new dataframes, and so on All polars methods are pure, e.g. they don't mutate the original.
- jiggunjer 4y ago
  You mean if your memory can only hold a single large dataframe, polars won't work well?
  - ayembee 4y ago
    No; there is an OOC (out-of-core, eg: > RAM) streaming mode. https://github.com/pola-rs/polars#handles-larger-than-ram-data https://github.com/pola-rs/polars#handles-larger-than-ram-da...
pydry 4y ago
The problem isnt really with the paradigm but that it is untyped. There are like 10 3rd party solutions for this but ffs it should be built in.
deleted 4y ago
[deleted]

ritchie46 4y ago

> Columns have names, except when they don't. Things (rows? individual cells?) They are not a fit for every problem. Please don't try to use them for that. For normalized tabular data, it is a good fit. > have types which is sometimes enforced and sometimes not. If you don't follow a recipe exactly, your dataframe will be subtly different and might not work. Polars is really strict on types. A type is always determined by the input datatype and the operation applied. You can check the schema before you run the query. Polars will produce the same output datatype as defined by its logical plan. > A lot of code mixes mutation in place and creating new dataframes, and so on All polars methods are pure, e.g. they don't mutate the original.

jiggunjer 4y ago
You mean if your memory can only hold a single large dataframe, polars won't work well?
- ayembee 4y ago
  No; there is an OOC (out-of-core, eg: > RAM) streaming mode. https://github.com/pola-rs/polars#handles-larger-than-ram-data https://github.com/pola-rs/polars#handles-larger-than-ram-da...

pydry 4y ago

The problem isnt really with the paradigm but that it is untyped. There are like 10 3rd party solutions for this but ffs it should be built in.

deleted 4y ago

[deleted]