7 ms·
I'm curious - what were you doing that polars was leaving a 40-80x speedup on the table? I've been happy with it's speed when held correctly, but it's certainly
by ladberg 7mo ago
I'm curious - what were you doing that polars was leaving a 40-80x speedup on the table? I've been happy with it's speed when held correctly, but it's certainly easy to hold it incorrectly and kill your perf if you're not careful
- devnotes77 7mo ago[flagged]
- __mharrison__ 7mo agoApp is now lazy!
- deleted 7mo ago[deleted]
- dartharva 7mo agoMight be tangential but in my recent experience polars kept crashing the python server with OOM errors whenever I tried to stream data from and into large parquet files with some basic grouping and aggregation. Claude suggested to just use DuckDB instead and indeed, it made short work of it.
- __mharrison__ 7mo ago20 year old BI app. Columnar DBs weren't really a thing. (MonetDB was brand new but not super stable. I committed the SQLAlchemy interface to it.)
- beagle3 7mo agoKDB v1 is from sometime in the late 1990’s (I met v2 in 2002; but v1 was internal use only at some investment bank). But that follows A and A+ which were extremely column oriented and date to early 1990s or even late 1980s ; and to various APL implementations going back to the 1960’s Columnar DBs were very much a thing among APL users (finance and operations research) but weren’t really known outside those fields - and even in those fields, there was a period of amnesia in the late ‘90s/early 2000’s