5 ms·

Similar experience to vulkoingim: steep learning curve but quite stable once deployed properly. Schema management in zookeeper has been the biggest pain point

Similar experience to vulkoingim: steep learning curve but quite stable once deployed properly. Schema management in zookeeper has been the biggest pain point for us. Occasionally individual clickhouse shards will get out of sync during a schema update, which can be hard to diagnose. We use a heavily modified version of clickhouse-backup[1], which works well for us. As for hands-off replica reboot: you must have an automated process to reapply the same schema which exists in zookeeper, otherwise it won't resync. If the local schema gets out of sync with that in zookeeper, then you'll have issues again. I expect a lot of these ergonomics issues will be fixed over time. It's already much easier to use than it was 3 years ago, and even if progress on usability and reducing the learning curve is slow the database performance makes it worth it. [1] https://github.com/AlexAkulov/clickhouse-backup https://github.com/AlexAkulov/clickhouse-backup

FridgeSeal 5y ago
I hold out perpetual hope that ClickHouse will move to a Raft or similar based consensus model and drop the external dependence on ZooKeeper on day.
- tonyhb 5y ago
  This is happening: https://github.com/ClickHouse/ClickHouse/issues/15090 https://github.com/ClickHouse/ClickHouse/issues/15090 They're using NuRaft from ebay to drop zookeeper. As far as I remember, it's actually already merged and available to test. Performance is ~3x slower in terms of write as they're coordinating everything themselves, but it should be _much_ easier to maintain. I'm looking forward to this, too, and want to see more in terms of docs / support here :)
  - FridgeSeal 5y ago
    Woah! That’s really exciting, I’d definitely tolerate some write slow-down if it means not having to run ZK.
    - hodgesrm 5y ago
      It's going to get faster. That's why it's still experimental. ;)
- brodock 5y ago
  people are asking for consul support for many months... it seems they refactored the dependency on zookeeper to make it easier to use something else

FridgeSeal 5y ago

I hold out perpetual hope that ClickHouse will move to a Raft or similar based consensus model and drop the external dependence on ZooKeeper on day.

tonyhb 5y ago
This is happening: https://github.com/ClickHouse/ClickHouse/issues/15090 https://github.com/ClickHouse/ClickHouse/issues/15090 They're using NuRaft from ebay to drop zookeeper. As far as I remember, it's actually already merged and available to test. Performance is ~3x slower in terms of write as they're coordinating everything themselves, but it should be _much_ easier to maintain. I'm looking forward to this, too, and want to see more in terms of docs / support here :)
- FridgeSeal 5y ago
  Woah! That’s really exciting, I’d definitely tolerate some write slow-down if it means not having to run ZK.
  - hodgesrm 5y ago
    It's going to get faster. That's why it's still experimental. ;)
brodock 5y ago
people are asking for consul support for many months... it seems they refactored the dependency on zookeeper to make it easier to use something else