4 ms·

This blog post offers a nice summary: https://motherduck.com/blog/six-reasons-duckdb-slaps/ https://motherduck.com/blog/six-reasons-duckdb-slaps/

• nerdponx 4y ago
  The blog post doesn't really make a comparison between DuckDB and data frame libraries. It mentions that the DuckDB Python bindings can interoperate with Pandas, but it doesn't really explain why you would use DuckDB instead of Pandas, or Polars (which is both faster and more portable than Pandas).
  • simonw 4y ago
    Don't Polars and Pandas both require your entire data to fit in memory?
    • nerdponx 4y ago
      Pandas doesn't. Polars I think has some lazy-loading capability, but it's not the default mode of operation and I don't think it supports all features. If DuckDB doesn't, then that's a big advantage.
      • jcheng 4y ago
        I think you mean that Pandas does require your entire data to fit in memory? https://pandas.pydata.org/docs/user_guide/scale.html https://pandas.pydata.org/docs/user_guide/scale.html
        • nerdponx 4y ago
          Yes, thank you, that was a typo.
      • corford 4y ago
        Apparently duckdb paired with arrow lets you work performatively on bigger than memory parquet data (I haven't tried this though)
        • riezebos 4y ago
          Arrow seems to be the common denominator for larger than ram. Polars is completely built on arrow, so the same advantages should apply. https://github.com/pola-rs/polars#handles-larger-than-ram-data https://github.com/pola-rs/polars#handles-larger-than-ram-da...