I’m a big fan of DuckDB. However it is true that memory usage is not optimized right now — I’ve had trouble with very large groupbys even with pragmas on memory

I’m a big fan of DuckDB. However it is true that memory usage is not optimized right now — I’ve had trouble with very large groupbys even with pragmas on memory limits (but I’m talking huge limits like 250 GB RAM). I’ve had to remodel and break down my problem to work around these limitations. I believe it’s on the DuckDB roadmap to get better at memory use but apart from that it’s very very good at what it does — ie complex analytical queries. It vectorizes ops so everything is fast and efficient.