3 ms·
How do you guarantee ACID with Kafka being responsible for actually INSERT'ing into ClickHouse? Wouldn't it be less error prone to just use ClickHouse directly
by klaussilveira 2y ago
How do you guarantee ACID with Kafka being responsible for actually INSERT'ing into ClickHouse? Wouldn't it be less error prone to just use ClickHouse directly and their async inserts?
https://clickhouse.com/blog/asynchronous-data-inserts-in-clickhouse https://clickhouse.com/blog/asynchronous-data-inserts-in-cli...
- pancomplex 2y agoI am thinking about setting this up as as a configuration for the type of traffic that doesn't require Kafka. That being said, Kafka has in my experience come in super handy again and again, simply because it adds an incredible extra layer of fault tolerance when running at scale, including the ability to replay events, replicate, fail over, etc. I'd be nervous about letting the amount of throughput we receive directly interface to ClickHouse (though I'd be excited to run an experiment with this).