3 ms·
Clickhouse or DuckDB are databases I would look at that support this use case pretty much "out of the box" E.g. https://benchmark.clickhouse.com https://benchm
by tveita 4y ago
Clickhouse or DuckDB are databases I would look at that support this use case pretty much "out of the box"
E.g. https://benchmark.clickhouse.com https://benchmark.clickhouse.com has some query times for a 100 million row dataset.
- spaniard89277 4y agoDuckDB is so simple to work with. It's only worth to look elsewhere with real big data, or where you really need a client-server setup. I hope it receives more love.
- IanCal 4y agoDuckdb is outrageously useful. Great on its own, but slots in perfectly reading and providing back arrow data frames, meaning you can seamlessly swap between tools when SQL for some parts and other tools better for others. Also very fast. I was able to throw away designs for multi machine setups as duckdb on its own was fast enough to not worry about anything else.
- intelVISA 4y agoHaving used all three I'd go with Clickhouse/DuckDB over Arrow every time.
- sanderjd 4y agoOh interesting - why?
- intelVISA 4y agoThey're easier to use and faster is the tl;dr.
- nlittlepoole 4y ago100% agree.
- lmeyerov 4y agoProbably for SQL (top n, ...), but not for wrangling & analytics & ML & ai & viz