4 ms·

Which is why postmortem blog posts are always infinity more enlightening than the "How we build X" posts

barrkel 6y ago
"All happy families are alike; each unhappy family is unhappy in its own way." https://en.wikipedia.org/wiki/Anna_Karenina_principle https://en.wikipedia.org/wiki/Anna_Karenina_principle
- WJW 6y ago
  Not to knock on Tolstoy but there are many ways in which unhappy families can be grouped together. You have the alcoholic parents, the living-vicariously-through-their-children folks, the abusive parents, etc etc etc. To tie it back to Kubernetes, you have the scheduler-does-not-schedule lessons, the ingress-not-consistent lessons, the autoscaler-killed-all-my-pods lessons, the moving-stateful-services-with-60TB-storage-size-took-20-minutes lessons and probably many more. It's like Churchills explanation of democracy: terrible, but better than all the alternatives. (at scale, at least)
  - teach 6y ago
    I guess it depends on your definition of "at scale" but IMO with Hashicorp's stuff and maybe something like Backstage you get all the benefits k8s but in a package that's much simpler to reason about and manage.
    - corty 6y ago
      By building a neat box with few buttons on the outside and still the same or more complexity on the inside. If something goes wrong, you still have to deal with all the inside parts. The only hope is that the few outside buttons only trigger happy paths without complications. "At scale" this seems unlikely, simplifications usually only work in smaller, simpler use cases.
      - jen20 6y ago
        Nomad very definitely does not beget the same gordion knot of complexity of Kubernetes, while working at substantially larger cluster sizes than Kubernernetes is capable of.
  - MichaelMoser123 6y ago
    it's the too many moving parts and black boxes blues.
  - nyellin 6y ago
    The problem is that everyone needs to learn those lessons on their own. (But I'm biased: I'm working on a troubleshooting tool for K8s which has remediation rules for all those common cases.)
    - amatix 6y ago
      The tool sounds interesting — can you share any more details?
  - wildermuthn 6y ago
    I think you inadvertently explained why Tolstoy is right: while there are many categories of failure, within those categories each failure is it’s own big, and even for the same bug there are often contextual reasons why the bug manifests in one scenario but not another. Yes, there are alcoholics, but every alcoholic has their own lonely story of descent into addiction. There are infinite ways to fail, and only finite ways to succeed. That relative difference between infinite and finite is what Tolstoy is getting at. And this is a helpful perspective, because then success becomes less about trying to prevent infinite failure-points and more about doing a few finite things very well. This is why you see advice from YC like — “don’t worry about competition”, and “talk to users, talk to users, talk to users.”

"All happy families are alike; each unhappy family is unhappy in its own way." https://en.wikipedia.org/wiki/Anna_Karenina_principle https://en.wikipedia.org/wiki/Anna_Karenina_principle

WJW 6y ago
Not to knock on Tolstoy but there are many ways in which unhappy families can be grouped together. You have the alcoholic parents, the living-vicariously-through-their-children folks, the abusive parents, etc etc etc. To tie it back to Kubernetes, you have the scheduler-does-not-schedule lessons, the ingress-not-consistent lessons, the autoscaler-killed-all-my-pods lessons, the moving-stateful-services-with-60TB-storage-size-took-20-minutes lessons and probably many more. It's like Churchills explanation of democracy: terrible, but better than all the alternatives. (at scale, at least)
- teach 6y ago
  I guess it depends on your definition of "at scale" but IMO with Hashicorp's stuff and maybe something like Backstage you get all the benefits k8s but in a package that's much simpler to reason about and manage.
  - corty 6y ago
    By building a neat box with few buttons on the outside and still the same or more complexity on the inside. If something goes wrong, you still have to deal with all the inside parts. The only hope is that the few outside buttons only trigger happy paths without complications. "At scale" this seems unlikely, simplifications usually only work in smaller, simpler use cases.
    - jen20 6y ago
      Nomad very definitely does not beget the same gordion knot of complexity of Kubernetes, while working at substantially larger cluster sizes than Kubernernetes is capable of.
- MichaelMoser123 6y ago
  it's the too many moving parts and black boxes blues.
- nyellin 6y ago
  The problem is that everyone needs to learn those lessons on their own. (But I'm biased: I'm working on a troubleshooting tool for K8s which has remediation rules for all those common cases.)
  - amatix 6y ago
    The tool sounds interesting — can you share any more details?
- wildermuthn 6y ago
  I think you inadvertently explained why Tolstoy is right: while there are many categories of failure, within those categories each failure is it’s own big, and even for the same bug there are often contextual reasons why the bug manifests in one scenario but not another. Yes, there are alcoholics, but every alcoholic has their own lonely story of descent into addiction. There are infinite ways to fail, and only finite ways to succeed. That relative difference between infinite and finite is what Tolstoy is getting at. And this is a helpful perspective, because then success becomes less about trying to prevent infinite failure-points and more about doing a few finite things very well. This is why you see advice from YC like — “don’t worry about competition”, and “talk to users, talk to users, talk to users.”