6 ms·

I use a lot of this distributed systems tooling on a day-to-day basis: Docker, Kubernetes, Vault, Pachyderm, etc., etc. It's all brilliant and innovative softwa

• bmurphy1976 10y ago
  > But every time I need to dig down into the source code, I quickly realize why Docker falls over all the time with weird bugs, and why Vault's high-availability backend for etcd falls over once per week, and why Kubernetes is capable of inspiring such epic love-hate relationships. Vault's documentation clearly states to not use ETCD as the HA backend. Docker's problems stem from their bone-headed decision to start with a client/server architecture that they've been unwinding ever since. They should have started with small composable tools that integrated well with the existing infrastructure (init systems, file distribution, and security infrastructure). Instead they started with an architecture where everything goes over an HTTP socket through a daemon with high probability for contention, deadlock, and failure. None of that has anything to do with Go. They could have built that abomination in any language. I can't speak for Kubernetes. I have not used it in anger yet.
  • ekidd 10y ago
    > Vault's documentation clearly states to not use ETCD as the HA backend. Yes, when I reported bugs against the etcd HA backend they said they were going to add more prominent warnings to the docs, if I remember correctly. :-) I think the underlying bug comes from overly optimistic multithreading code using CSP, but I stared at the code for several hours and couldn't see how to make it unambiguously correct. We now have our own in-house Redis "HA" backend that I wrote which appears to be rock solid in production, but which temporarily disables Vault during Redis restarts. So it's really "fake HA" but it works nicely for our use case and never requires manual intervention or embarrasses us during business hours. I refuse to set up and administer an entire Consul cluster for a single leader-election lock. Managing Consul has its own complications: https://www.consul.io/docs/guides/outage.html https://www.consul.io/docs/guides/outage.html > Instead they started with an architecture where everything goes over an HTTP socket through a daemon with high probability for contention, deadlock, and failure. None of that has anything to do with Go. The Docker daemon's wire protocol suffers from a kind a of sloppy thinking about data types, in my opinion. Consider this line in a Rust Docker client: https://github.com/faradayio/boondock/blob/bf29afb0c78f8d5d6e5bf8dad56af7eeebd94935/src/container.rs#L123 https://github.com/faradayio/boondock/blob/bf29afb0c78f8d5d6... pub Ports: Option<HashMap<String, Option<Vec<PortMapping>>>>, Here, the `Option` types say, "This might be a JSON `null` value on the wire." So the `Ports` member might be `null`, or it might contain an empty hashmap, which in turns contains values that might be `null`, or might be `PortMapping` value. The corresponding Go code is extremely unclear (at least in most similar situations) as to which values can be `null` on the wire. There's a lot of really foggy specifications going on in the network layer of the Docker server. It's not bad in the way that certain old Unix servers like `csvd` were bad, but I feel like Docker could be better if Go demanded more clarity about data types. On the other hand, Docker is clearly a raging commercial success, and it actually works quite nicely (at least if you use Amazon's ECS to manage your clusters). So maybe I'm just being overly fastidious. Clearly all this new Go-based devops infrastructure is hugely successful, even if many of us get frustrated with it now and then. Still, I really wish that more programming languages forced you to be clear about which values can be `null` and which can't. Hoare called it his "billion dollar mistake": http://lambda-the-ultimate.org/node/3186 http://lambda-the-ultimate.org/node/3186
• lobster_johnson 10y ago
  I have yet to see Go code among that class of projects that I would consider bad. I find Go to be a kind of equalizer -- the low ceiling prevents masters from moving upwards in syntactical/semantic complexity, and the high, uh, floor prevents junior developers from doing too much obviously bad. There are plenty of edge cases a newbie can cut themselves on (the addition of unsafe pointers is an antique, Victorian footgun that you'd not expect in a modern language), but overall, Go's forced simplicity normalizes code quality to a degree that I've not seen elsewhere. Docker's faults do not come from Go, and the same goes for Kubernetes.
• geodel 10y ago
  Well I do hope analogous tools appear in Rust and show design soundness of Rust. For now I remain sceptic that Rust is perfect language for large, complex, high-perf highly concurrent software. Rust Roadmap says this much for production use: "Production use matters for the obvious reason: it grows the set of stakeholders with potential to invest in the language and ecosystem. To deliver on that potential, Rust needs to be part of the backbone of some major products. Production use measures our design success; it's the ultimate reality check. Rust takes a unique stance on a number of tradeoffs, which we believe to position it well for writing fast and reliable software. The real test of those beliefs is people using Rust to build large, production systems, on which they're betting time and money. The kind of production use matters. For Rust to truly be a success, there should be clear-cut reasons people are employing it rather than another language. Rust needs to provide crisp, standout benefits to the organizations using it." https://github.com/aturon/rfcs/blob/roadmap-2017/text/0000-roadmap-2017.md https://github.com/aturon/rfcs/blob/roadmap-2017/text/0000-r...