6 ms·
A tangent, from someone still doing most of their mangling the old fashioned way in sql used to extract stuff into old fashioned pandas df in a notebook, ... bu
by willvarfar 3y ago
A tangent, from someone still doing most of their mangling the old fashioned way in sql used to extract stuff into old fashioned pandas df in a notebook, ... but I'm wondering and HN is a wonderful place to ask :)
How does polars sql context stack up against alternatives e.g. perhaps duckdb? If I'm in a notebook and I want to suck in and process a lot of data, which has the least boilerplate, the strongest support and the most efficiency (both RAM usage and speed)?
- lordgrenville 3y agoHere is a head-to-head comparison about efficiency: https://www.youtube.com/watch?v=wKH0-zs2g_U https://www.youtube.com/watch?v=wKH0-zs2g_U "Strongest support" is probably Pandas, in that it is very widely used and easy to get help with. DuckDB lets you write SQL and is very fast.
- ritchie46 3y agoThat comparison is heavily outdated as at that benchmark we are IO bound on downloading. Since then Polars has improved downloading speeds 20x with shipping a proper async runtime in the engine.