4 ms·

We use consul a bit as some "light" service discovery and a KV store for a few things without much issue so far. What demons did you encounter with vault/consu

• hughrr 5y ago
  We had some massive problems including complete cluster collapse requiring rebuilds from scratch, eternal leadership elections and occasionally nodes would just entirely stop responding to KV requests causing cascading failures outside. Vault is a massive damage multiplier for these issues plus some other nasty ones like buggy barely supported plugins.
  • zetsurin 5y ago
    Wouldn't mind knowing what versions you were using/how long ago this was. We haven't seen anything like this in > 3 years. Possible a barely supported plugin caused issues?
    • hughrr 5y ago
      This is post 1.0 and without any plugins for consul. Vault has been a persistent point of failure. It’s a good idea in principle but it’s not got a good ROI
      • chucky_z 5y ago
        Huh. I've been running Consul since 0.7 or so, and the only time any of this happened was my own fault. Most of the problems I've had with Vault have been around it's Terraform provider which they've improved enough that it's not an issue anymore. I think the only thing about Raft that folks don't realize is how disk hungry it gets, if you want fast write performance you gotta make dang sure all those fsyncs can keep up. Our largest Consul cluster today runs on storage-heavy boxes as it does ~500Mb/s of writes pretty much 24/7. I use Consul w/ Vault today instead of the internal storage for Vault just cause Consul has really nice monitoring around some stuff that Vault doesn't (path-based stuff for the most part), I think the internal storage is a really good option for 90% of use-cases.