4 ms·

People jump on bandwagons. Ten years ago you started with PHP + memcache + MySQL running on just a physical box running linux. No Docker or Kubernetes. No virt

People jump on bandwagons. Ten years ago you started with PHP + memcache + MySQL running on just a physical box running linux. No Docker or Kubernetes. No virtual machine for what it is worth. Then you split it in multiple PHP frontends with a load balancer or MySQL master and slaves as the traffic demanded it. I think a lot of systems these days are over-engineered and over-paid from day one. It may cost you 10-20 times what you would if you keep the "old day" approach. That said Docker and Kubernetes are nice. They give you a lot of flexibility. But the most important part, I think, has been that they consolidated the shift from "pet" to "cattle" servers that simplifies a lot the sysOp and sysAdm work.

moksly 7y ago
I don’t think kubernetes is nice because I’ve seen more than one small team throw a lot of resources at it and fail. I’m not attacking docker, however, I can see why you would want containers. We still haven’t found an efficient usage for them at my place, but we never need to spin up more than one instance of our software. I think docker can sometimes be a way to cheat unsafe software around operations, but that’s more of an anti-pattern than an issue with docker. Then again, I’m probably old and grumpy, but that sometimes has the advantage of not adopting techs before they are easy to use.
- akoncius 7y ago
  could you elaborate more about failure of kubernetes? I'm considering using docker/kubernetes in our production so to hear warning stories would be extremely helpful.
geggam 7y ago
Disclaimer : I get paid to setup k8s clusters for the bandwagon folks From my perspective k8s is really good for a company who has a team of experienced SRE who can manage k8s well and the company has a marketing team driving new development that needs to get to market fast It is not cheaper than simple autoscale groups, it is not easier than rebuilding packages with jenkins ( rpm spec files or debian src rebuilds ) It is however the current popular framework so if you want to ride the wave learn it. Also learn how to migrate away from it as that will be a future role as several early adopters are pulling back out of it. The larger issue is the simple fact automating cloud infrastructure is not trivial so abstract layers lets more people implement things without understanding the lower levels giving folks the appearance of having a complete framework. It works great until it doesn't. Then it is interesting watching people try to figure out how to fix it. Learn the lower levels of Linux and k8s will come to you organically
- cpb 7y ago
  In your experience, what is the strongest point of evidence leading to the successful utilization of k8s? > the company has a marketing team driving new development that needs to get to market fast At what point in the growth curve? Are your SRE shipping the new features, or running like a red queen to keep the product developers able to keep shipping? What development process or cycle has overheated with friction that requires k8s? Where is that friction which k8s relieves?
  - geggam 7y ago
    :) Successful utilization of k8s, interesting metric, what is your criteria ? >At what point in the growth curve? I have seen properly staffed startups (in silicon valley) leverage the platform to pivot direction fast but overall I would say you need a large enterprise to support it. Ironically the large enterprise which could benefit the most wants to lay ITIL on top of k8s, call it agile and the methodologies conflict creating more issues than they had. > Are your SRE shipping the new features, or running like a red queen to keep the product developers able to keep shipping? Yes to both >What development process or cycle has overheated with friction that requires k8s? Upgrading k8s :) >Where is that friction which k8s relieves? What k8s relieves is once you get a pipeline getting product to production fast, the approval process from legacy ITIL groups being the problem ( I have yet to see an enterprise company use k8s properly though I have been to presentations where some claim to have done this ) If someone attempts to use ITIL and k8s you are going to have problems. Training people to NOT do this is the #1 issue in my experience.
    - cpb 7y ago
      All my mentions of issues or friction in my previous comment was about pre-k8s team's experiences... I try and clarify a few key cases in the following: > Successful utilization of k8s, interesting metric, what is your criteria ? Sorry, what pain motivated using k8s, and using k8s relieved that pain. > I have seen properly staffed startups (in silicon valley) What is 'proper' for staffing? >> Are your SRE shipping the new features, or running like a red queen to keep the product developers able to keep shipping? > Yes to both Is it the appropriate use of a Reliability engineer's skills to develop or change arbitrary features? Or are you saying they are at least Engineers so they should be able to pitch in everywhere... CSS accessibility features or k8s config. >> What development process or cycle has overheated with friction that requires k8s? > Upgrading k8s :) This tautology keeps me away from k8s. Before k8s what pain in the process required using k8s to solve? > a pipeline getting product to production fast Does k8s make it fast? Does k8s make it possible? I think I'm deploying "fast", without k8s... But because you keep mentioning ITIL I suppose it's more about the infrastructure changes? Or are you talking about ITIL tension because k8s requires a more liberal policy than ITIL allows? There is so much assumed context when talking about k8s, the only comprehensible part of these discussions is k8s is a rabbit hole to end all rabbit holes.

moksly 7y ago

I don’t think kubernetes is nice because I’ve seen more than one small team throw a lot of resources at it and fail. I’m not attacking docker, however, I can see why you would want containers. We still haven’t found an efficient usage for them at my place, but we never need to spin up more than one instance of our software. I think docker can sometimes be a way to cheat unsafe software around operations, but that’s more of an anti-pattern than an issue with docker. Then again, I’m probably old and grumpy, but that sometimes has the advantage of not adopting techs before they are easy to use.

akoncius 7y ago
could you elaborate more about failure of kubernetes? I'm considering using docker/kubernetes in our production so to hear warning stories would be extremely helpful.

geggam 7y ago

Disclaimer : I get paid to setup k8s clusters for the bandwagon folks From my perspective k8s is really good for a company who has a team of experienced SRE who can manage k8s well and the company has a marketing team driving new development that needs to get to market fast It is not cheaper than simple autoscale groups, it is not easier than rebuilding packages with jenkins ( rpm spec files or debian src rebuilds ) It is however the current popular framework so if you want to ride the wave learn it. Also learn how to migrate away from it as that will be a future role as several early adopters are pulling back out of it. The larger issue is the simple fact automating cloud infrastructure is not trivial so abstract layers lets more people implement things without understanding the lower levels giving folks the appearance of having a complete framework. It works great until it doesn't. Then it is interesting watching people try to figure out how to fix it. Learn the lower levels of Linux and k8s will come to you organically

cpb 7y ago
In your experience, what is the strongest point of evidence leading to the successful utilization of k8s? > the company has a marketing team driving new development that needs to get to market fast At what point in the growth curve? Are your SRE shipping the new features, or running like a red queen to keep the product developers able to keep shipping? What development process or cycle has overheated with friction that requires k8s? Where is that friction which k8s relieves?
- geggam 7y ago
  :) Successful utilization of k8s, interesting metric, what is your criteria ? >At what point in the growth curve? I have seen properly staffed startups (in silicon valley) leverage the platform to pivot direction fast but overall I would say you need a large enterprise to support it. Ironically the large enterprise which could benefit the most wants to lay ITIL on top of k8s, call it agile and the methodologies conflict creating more issues than they had. > Are your SRE shipping the new features, or running like a red queen to keep the product developers able to keep shipping? Yes to both >What development process or cycle has overheated with friction that requires k8s? Upgrading k8s :) >Where is that friction which k8s relieves? What k8s relieves is once you get a pipeline getting product to production fast, the approval process from legacy ITIL groups being the problem ( I have yet to see an enterprise company use k8s properly though I have been to presentations where some claim to have done this ) If someone attempts to use ITIL and k8s you are going to have problems. Training people to NOT do this is the #1 issue in my experience.
  - cpb 7y ago
    All my mentions of issues or friction in my previous comment was about pre-k8s team's experiences... I try and clarify a few key cases in the following: > Successful utilization of k8s, interesting metric, what is your criteria ? Sorry, what pain motivated using k8s, and using k8s relieved that pain. > I have seen properly staffed startups (in silicon valley) What is 'proper' for staffing? >> Are your SRE shipping the new features, or running like a red queen to keep the product developers able to keep shipping? > Yes to both Is it the appropriate use of a Reliability engineer's skills to develop or change arbitrary features? Or are you saying they are at least Engineers so they should be able to pitch in everywhere... CSS accessibility features or k8s config. >> What development process or cycle has overheated with friction that requires k8s? > Upgrading k8s :) This tautology keeps me away from k8s. Before k8s what pain in the process required using k8s to solve? > a pipeline getting product to production fast Does k8s make it fast? Does k8s make it possible? I think I'm deploying "fast", without k8s... But because you keep mentioning ITIL I suppose it's more about the infrastructure changes? Or are you talking about ITIL tension because k8s requires a more liberal policy than ITIL allows? There is so much assumed context when talking about k8s, the only comprehensible part of these discussions is k8s is a rabbit hole to end all rabbit holes.