6 ms·

It uses Raft underneath as well which means there's a bunch of non-determinism and hell for anyone who invokes it as well from experience. The thing is cursed.

• cortesoft 5y ago
  What is a better consensus protocol to use?
  • hughrr 5y ago
    Look higher up the problem domain and solve it without requiring a consensus protocol.
    • cortesoft 5y ago
      Hmm, how do you have high availability consistent data without a consensus protocol? No matter where in the problem chain you move, you have to eventually solve that problem.
      • hughrr 5y ago
        Easy you push the configuration to every target. Then they are logically consistent. We did this for 30 years fine before someone invented this stack on deployments much larger then the average consul or vault deployment these days. I had something running 15,000 dynamic rps on Apache about 15 years ago. People are blinded from simplicity by complexity. Eventually complexity owns you. You can only own simplicity. At the end of the day this is one way to solve a problem that doesn’t need to be solved that someone has convinced you is a problem.
        • asadawadia 5y ago
          >Easy you push the configuration to every target. Then they are logically consistent. What does this mean? This doesn't mean anything? Are you saying to push the DB to the client?
          • caffeine 5y ago
            I think he’s saying you have a cronjob with an rsync to every client
        • cortesoft 5y ago
          People need distributed consistent data for a lot of things besides configuration.
          • yencabulator 5y ago
            People say they do, but then they proceed to read stale data from replicas all the time...
    • rubiquity 5y ago
      Not sure why you’re being down voted as this is definitely the path forward. The industry and research have both explored the polar opposites of weak consistency protocols and linearizibility and consensus on the other end. Understanding your domain and knowing how you can step down to something weaker than consensus will be critical for future applications.
• tempest_ 5y ago
  We use consul a bit as some "light" service discovery and a KV store for a few things without much issue so far. What demons did you encounter with vault/consul?
  • hughrr 5y ago
    We had some massive problems including complete cluster collapse requiring rebuilds from scratch, eternal leadership elections and occasionally nodes would just entirely stop responding to KV requests causing cascading failures outside. Vault is a massive damage multiplier for these issues plus some other nasty ones like buggy barely supported plugins.
    • zetsurin 5y ago
      Wouldn't mind knowing what versions you were using/how long ago this was. We haven't seen anything like this in > 3 years. Possible a barely supported plugin caused issues?
      • hughrr 5y ago
        This is post 1.0 and without any plugins for consul. Vault has been a persistent point of failure. It’s a good idea in principle but it’s not got a good ROI
        • chucky_z 5y ago
          Huh. I've been running Consul since 0.7 or so, and the only time any of this happened was my own fault. Most of the problems I've had with Vault have been around it's Terraform provider which they've improved enough that it's not an issue anymore. I think the only thing about Raft that folks don't realize is how disk hungry it gets, if you want fast write performance you gotta make dang sure all those fsyncs can keep up. Our largest Consul cluster today runs on storage-heavy boxes as it does ~500Mb/s of writes pretty much 24/7. I use Consul w/ Vault today instead of the internal storage for Vault just cause Consul has really nice monitoring around some stuff that Vault doesn't (path-based stuff for the most part), I think the internal storage is a really good option for 90% of use-cases.
• PYTHONDJANGO 5y ago
  Please post URLs to bugs / issues to give your comment some cred. Thanks!