4 ms·
Correct! Currently no batching but no need to make things complicated for now. Can extend the current redis queue to batch request or clickhouse batch engine.
by lindesvard 2y ago
Correct! Currently no batching but no need to make things complicated for now.
Can extend the current redis queue to batch request or clickhouse batch engine.
Looking forward to have these problems hehe.
Please DM if you want to chat about best approach!
- klaussilveira 2y agoWon't this be a problem as soon as a website reaches, say, 600 RPS? Which is not much.
- lindesvard 2y agoI have time to grow. Don’t think I’ll hit this level just yet.
- FridgeSeal 2y agoAs someone using clickhouse, for similar event ingestion stuff - I’d lean towards implementing it sooner rather than later. It’s not too hard, and it can surprising _how much more_ Clickhouse likes bulk queries.
- lindesvard 2y agoSure thing! Will look into it more during the coming weeks. Feels like I have the most of the dashboard complete now. But I really want to start giving the native app some love as well
- jack_riminton 2y agoThis is the correct answer! any early project needs to make big trade-offs and premature optimisation has killed many projects. You'll never please everyone (especially on HN) but you're not making it for everyone
- Aeolun 2y agoDunno, that’s already a fairly big website. The one I was working on in the past with 15k customers did around 15RPS. The large enterprise system I work on with 3k users does slightly less than that. That covers an awfully large amount of use cases already, and then you can get 30x more?
- Yanyuk 2y agoOr enable async inserts on the ClickHouse client, which won't require changing to the buffer table engine. https://clickhouse.com/docs/en/optimize/asynchronous-inserts https://clickhouse.com/docs/en/optimize/asynchronous-inserts
- lindesvard 2y agoHave read about this, but totally forgot about it! Will try it out!