4 ms·

Too bad polars like everything else written in rust has horrible ergonomics. I don’t know how anyone can look at the polars rust API and say in good faith that

Too bad polars like everything else written in rust has horrible ergonomics. I don’t know how anyone can look at the polars rust API and say in good faith that not having default arguments and named parameters was a good idea.

aguspdana 4y ago
I agree that the Rust API is still rough. But.. Polars has Python API. It's much nicer than the Rust API. Plus the documentation is more complete.
- Laiho 4y ago
  Couldn't agree more about the rust API. Python API seems OK. Found it very complicated to use and just clunky in general.
wenc 4y ago
Although Polars is written in Rust, most people will use Polars from Python since that what most data wranglers use. And Polars’ Python interface is excellent.
- detrites 4y ago
  Do you happen to know if this can be used as more or less a drop-in replace for pandas in Python? (Presuming, I suppose data conversion prior.)
  - wenc 4y ago
    It depends on what you do with Pandas. The semantics are different so you’ll have to rewrite stuff — that said, for me it was worth it for the most part because I work with massive columnar data in Parquet. So no, not a drop in replacement. But not a difficult transition either. This page explains how Polars differs from Pandas. https://pola-rs.github.io/polars-book/user-guide/coming_from_pandas.html https://pola-rs.github.io/polars-book/user-guide/coming_from...
    - valarauko 4y ago
      As someone who loathes the Pandas syntax and lusts for the relatively cleaner tidyverse code of my colleagues, the Polar syntax just feels ... off (from the link): df.with_columns( pl.when(pl.col("c") == 2) .then(pl.col("b")) .otherwise(pl.col("a")).alias("a") ) Seeing the multiple nested pl calls within a single expression just feels odd to me. It's definitely reminiscent of Dplyr but in a much less elegant way.
      - dunefox 4y ago
        Yeah, I tried to get into it but it seems very verbose.
      - wenc 4y ago
        You can use DuckDB (SQL syntax) then just convert to Polars (instantaneous). The method chaining syntax is unwieldy in any language. Magrittr + dplyr (tidyverse) pipeline syntax is beautiful syntax but there’s a lot of magic with NSE (nonstandard evaluation) that makes it really tricky when you need to pass a variable column name. I’ve sort of converged on SQL as the best compromise.
        valarauko 4y ago
        I like the idea of Polars, and it's on my list of things to try. Unfortunately, my current codebase in heavily tied to Pandas (bioinformatics tools that use Pandas as a dependency). I also deal mostly in sparse matrices, which I'm unsure if duckdb handles. Looking at the Polars documentation also makes me nervous, due to how much of my current Pandas-fu relies on indexing to work. I appreciate that indexes can be NSE but it's how a lot of the current tools in my field work (python and in R) with important data in the index, eg, genes or cellular barcodes, and relying on the index to merge datasets. Another caveat for me is at multiple times in my workflow I drop into R for plots and rpy2 can convert pandas dataframes into equivalent R atomics. With Polars it would be just an additional step of converting to pandas df but just something I need to consider. That said, I've disliked the Pandas syntax for so long that the mental overhead might be well worth it.
    - 4y ago

aguspdana 4y ago

I agree that the Rust API is still rough. But.. Polars has Python API. It's much nicer than the Rust API. Plus the documentation is more complete.

Laiho 4y ago
Couldn't agree more about the rust API. Python API seems OK. Found it very complicated to use and just clunky in general.

wenc 4y ago

Although Polars is written in Rust, most people will use Polars from Python since that what most data wranglers use. And Polars’ Python interface is excellent.

detrites 4y ago
Do you happen to know if this can be used as more or less a drop-in replace for pandas in Python? (Presuming, I suppose data conversion prior.)
- wenc 4y ago
  It depends on what you do with Pandas. The semantics are different so you’ll have to rewrite stuff — that said, for me it was worth it for the most part because I work with massive columnar data in Parquet. So no, not a drop in replacement. But not a difficult transition either. This page explains how Polars differs from Pandas. https://pola-rs.github.io/polars-book/user-guide/coming_from_pandas.html https://pola-rs.github.io/polars-book/user-guide/coming_from...
  - valarauko 4y ago
    As someone who loathes the Pandas syntax and lusts for the relatively cleaner tidyverse code of my colleagues, the Polar syntax just feels ... off (from the link): df.with_columns( pl.when(pl.col("c") == 2) .then(pl.col("b")) .otherwise(pl.col("a")).alias("a") ) Seeing the multiple nested pl calls within a single expression just feels odd to me. It's definitely reminiscent of Dplyr but in a much less elegant way.
    - dunefox 4y ago
      Yeah, I tried to get into it but it seems very verbose.
    - wenc 4y ago
      You can use DuckDB (SQL syntax) then just convert to Polars (instantaneous). The method chaining syntax is unwieldy in any language. Magrittr + dplyr (tidyverse) pipeline syntax is beautiful syntax but there’s a lot of magic with NSE (nonstandard evaluation) that makes it really tricky when you need to pass a variable column name. I’ve sort of converged on SQL as the best compromise.
      - valarauko 4y ago
        I like the idea of Polars, and it's on my list of things to try. Unfortunately, my current codebase in heavily tied to Pandas (bioinformatics tools that use Pandas as a dependency). I also deal mostly in sparse matrices, which I'm unsure if duckdb handles. Looking at the Polars documentation also makes me nervous, due to how much of my current Pandas-fu relies on indexing to work. I appreciate that indexes can be NSE but it's how a lot of the current tools in my field work (python and in R) with important data in the index, eg, genes or cellular barcodes, and relying on the index to merge datasets. Another caveat for me is at multiple times in my workflow I drop into R for plots and rpy2 can convert pandas dataframes into equivalent R atomics. With Polars it would be just an additional step of converting to pandas df but just something I need to consider. That said, I've disliked the Pandas syntax for so long that the mental overhead might be well worth it.
  - 4y ago