4 ms·

They mention https://pola.rs/ https://pola.rs/ and highlight how it has the exact same properties (out of core etc) but don't include it in benchmarks and no me

by mborch 3y ago

They mention https://pola.rs/ https://pola.rs/ and highlight how it has the exact same properties (out of core etc) but don't include it in benchmarks and no mention on their detailed comparison page.

wdroz 3y ago
It's because they are comparing to distributed solutions only. Polars is one of the fastest solution for single computer workflow, but it doesn't support distributed workflow.
- mborch 3y ago
  Ah of course, that makes sense, thanks. In their comparison, perhaps they should still mention that just to clear up any confusion.
  - jaychia 3y ago
    Oh yes good point! We'll be sure to add more details about comparisons with local dataframe libraries such as Pandas/Polars/DuckDB.
baggiponte 3y ago
I was told they use Polars engine; so they "just" build a distributed system around it
- wdroz 3y ago
  I don't see any direct dependencies to polars. But in the comment, they wrote that some part are "taken from", "influenced by", "adapted from" and "based on" polars. [0] -- https://github.com/search?q=repo%3AEventual-Inc%2FDaft%20polars&type=code https://github.com/search?q=repo%3AEventual-Inc%2FDaft%20pol...
  - baggiponte 3y ago
    yes exactly, they """took""" part of the Polars engine code.
- jaychia 3y ago
  Hello! Daft developer here - we don’t directly use Polars as an execution engine, but parts of the codebase (e.g. the expressions API) are heavily influenced by Polars code and hence you may see references to Polars in those sections. We do have a dependency on the Arrow2 crate like Polars does, but that has been deprecated recently so both projects are having to deal with that right now.