6 ms·

At this point, why not just drive it up to the logical conclusion? Treat your business model as cattle, not pets. Customers leaving? Fire up another business

At this point, why not just drive it up to the logical conclusion? Treat your business model as cattle, not pets. Customers leaving? Fire up another business until capital runs out, and if it does, no worries, jut hop to another job! Sorry, but I feel like I landed in crazy-land. Kubernetes is already an exercise in how many layers you can insert with nobody understanding the whole picture. Ostensibly, it's so that you can isolate those fucking jobs so that different teams can run different tasks in the same cluster without interfering with each other. Hence namespaces, services, resource requirements, port translation, autoscalers, and all those yaml files. It boggles my mind that people look at something like Kubernetes and decide "You know what? We need more layers. On top of this."

spondyl 5y ago
I mean, depending on the business, employees aren't trusted to understand the whole picture regardless. Many employees at traditional business don't even have administrator access on their laptops, depending on their position so it's not logically inconsistent with how things seem to operate. With that lack of big picture overview, it makes things hard to scrutinise since you're only seeing one sliver of an implementation (ie; "It saves money" without seeing, let alone understanding the technical requirements and vice versa)
- aynyc 5y ago
  Limited admin practice isn’t just to save money. It’s a security practice and it’s a good one. In a large business, no one understand the whole business maybe except legal department and CEO.
  - MuffinFlavored 5y ago
    > In a large business, no one understand the whole business maybe except legal department and CEO. lol... what? A CEO of a 20,000+ person company literally sees projects as revenue sources and a deadline, nothing more. To get it delivered, he/she walks down the chain of managers until they gets answers. It's might as well be a black hole.
    - aynyc 5y ago
      I only work for three large (2000+) companies. I’ve written reports to executive management committees. CEO never really cares about projects per se. they care about business units. Each business unit can have important projects. CEOs and management committees don’t really dive into them, they care about how each business unit are operating according to his management goals. CEOs aren’t gonna care about Python or Golang.
  - 908B64B197 5y ago
    > Limited admin practice isn’t just to save money. It’s a security practice and it’s a good one. And it's a red flag. Means that IT is running the show, not engineering.
  - spondyl 5y ago
    I agree in principle! To be a bit clearer, I was thinking more in terms of locality so for example, "Users don't even have access to their laptops right in front of them" let alone visibility into their own department or even wider organisation due to access policy restrictions. Being able to write or edit everywhere would be ridiculous but sometimes there are (admittedly intangible) upsides to transparency (read only access in a technical sense) to most aspects of the wider business
  - apetresc 5y ago
    You think the _legal department_ understands the whole business better than anyone? Where have you been working?
    - aynyc 5y ago
      Every company I work for, legal usually sign of every deal, in terms of legality of the contract. They know every contracts/clients we have. They know where all our business is. They might not care whether our business plan is or works, but they know what our business is. Same can be say about finance group.
- 908B64B197 5y ago
  Also, the caliber of engineers at Google might not be the same as RandomCo where admin access isn't allowed for engineers.
tyingq 5y ago
"You know what? We need more layers. On top of this." Heh. Service Mesh!
- rantwasp 5y ago
  K8s on K8s! K64s!
  - voidfunc 5y ago
    But can it run on an N64?
    - rantwasp 5y ago
      it starts at Nintendo 69.
  - yongjik 5y ago
    Kuberneteuberneteuberneteuberneteuberneteuberneteuberneteubernetes is a perfect name for the beautiful system you are about to architect: it's an ancient Greek word for helmsman-man-man-man-man-man-man-man, who is in charge of helmsman-man-man-man-man-man-man, who is in charge of ... (...sorry =_=)
  - verst 5y ago
    You joke, but have you looked at K8s VirtualCluster? Separate control planes for each tenant within a K8s cluster. https://github.com/kubernetes-sigs/cluster-api-provider-nested/tree/main/virtualcluster https://github.com/kubernetes-sigs/cluster-api-provider-nest...
    - rantwasp 5y ago
      of course. of course! it’s k8s clusters all the way down
  - lubujackson 5y ago
    "Nobody will ever need more than K64." - Robo-Gates
tene 5y ago
I'm in kind of the opposite position, as I feel confused about why I keep seeing so much resistance to higher levels of abstraction here. > It boggles my mind that people look at something like Kubernetes and decide "You know what? We need more layers. On top of this." I'm not involved with this project at all, but yes, I look at the massive Kubernetes deployments I've worked with, and conclude that toil and various other kinds of problems could be reduced with higher-level abstractions for declaratively managing configuration for all of these clusters. If you wanted to run workloads on 100k servers, how would you do it? Would you have a team manually configure each cluster individually, or would you consider that there are a lot of similarities between clusters, and you only want differences that are chosen intentionally, and look into some abstraction to help keep the complexity down? If 100k isn't enough for you, is there any number of nodes at which you'd consider building some tools support for your work? What about data center count and change rate? There are a lot of business models that benefit from running workloads closer to customers, so it's useful to have deployments in many datacenters and clouds across the world, and for this to grow over time. Do you really not see why some people are interested in being able to bring up a cluster in a new DC by updating a few configs rather than starting from nothing by hand with kubeadm init? I apologize if I've misrepresented your position by misunderstanding something. I feel like I'm missing or misunderstanding something from your perspective, but I don't know what it is. More generally, I think it's extremely normal for people to build higher levels of abstraction as things grow. It's usually worth considering abstraction or architectural changes in a system every time it grows by about 10x. Some of the most painful platforms I've supported in my career have been those that didn't do this, and instead tried to grow through brute force. Something can be fine to do once, troublesome to do 10 times, and overwhelming to do 100 times.
- smichel17 5y ago
  I think the comment you are replying to was objecting to the number of layers, rather than the level, of abstraction. More concretely, I'd guess they would not object to a tool that replaces k8s and offers a higher level api, but they do object to building said tool on top of k8s. For myself, I think it depends on how leaky the abstractions are. The leakier, the fewer layers the house of cards can stand. That is, it's more about where you slice it than how many times.
Groxx 5y ago
With a sufficiently-large N, N+1 is barely noticeable.
throwaway894345 5y ago
> Ostensibly, it's so that you can isolate those fucking jobs so that different teams can run different tasks in the same cluster without interfering with each other. I think the point is so the teams can run services at all. Isolation isn’t the point, it’s breaking a dependency on an ops team (Kubernetes in a sense is ops as a service).
jldugger 5y ago
> Fire up another business until capital runs out, and if it does, no worries, jut hop to another job! I hear people say Google runs their business this way. So perhaps it makes sense that Kubernetes came from them.
sangnoir 5y ago
> At this point, why not just drive it up to the logical conclusion? At this point, why not just drive it up to the logical conclusion? Treat your business model as cattle, not pets. We're already there. Cue "We invest in teams not ideas"[1,2] and "We're pivoting since we couldn't get a product-market match, but have millions in the bank"[3] 1.https://www.forbes.com/sites/ilyapozin/2015/10/08/4-reasons-to-invest-in-people-not-companies/?sh=4c8fd6bc47cb https://www.forbes.com/sites/ilyapozin/2015/10/08/4-reasons-... 2. https://www.gsb.stanford.edu/insights/do-funders-care-more-about-your-team-your-idea-or-your-passion https://www.gsb.stanford.edu/insights/do-funders-care-more-a... 3. https://triplebyte.com/blog/rethinking-triplebyte https://triplebyte.com/blog/rethinking-triplebyte
dexterous 5y ago
The only explanation I have that makes any sense is that it is a very Application Developer solution to a System Administration that hasn't matured it's abstractions and is now trapped in Tech Debt purgatory.
bradhe 5y ago
> It boggles my mind that people look at something like Kubernetes and decide "You know what? We need more layers. On top of this." This statement is less about adding another layer. It's more about not falling in to the same trap that we did with VMs in 2010--managing them by hand, producing unreproducible configurations, "tending" to them. The same pattern is happening with Kubernetes, but it fundamentally runs on computers at the end of the day so it's prone to similar types of failures.

spondyl 5y ago

I mean, depending on the business, employees aren't trusted to understand the whole picture regardless. Many employees at traditional business don't even have administrator access on their laptops, depending on their position so it's not logically inconsistent with how things seem to operate. With that lack of big picture overview, it makes things hard to scrutinise since you're only seeing one sliver of an implementation (ie; "It saves money" without seeing, let alone understanding the technical requirements and vice versa)

aynyc 5y ago
Limited admin practice isn’t just to save money. It’s a security practice and it’s a good one. In a large business, no one understand the whole business maybe except legal department and CEO.
- MuffinFlavored 5y ago
  > In a large business, no one understand the whole business maybe except legal department and CEO. lol... what? A CEO of a 20,000+ person company literally sees projects as revenue sources and a deadline, nothing more. To get it delivered, he/she walks down the chain of managers until they gets answers. It's might as well be a black hole.
  - aynyc 5y ago
    I only work for three large (2000+) companies. I’ve written reports to executive management committees. CEO never really cares about projects per se. they care about business units. Each business unit can have important projects. CEOs and management committees don’t really dive into them, they care about how each business unit are operating according to his management goals. CEOs aren’t gonna care about Python or Golang.
- 908B64B197 5y ago
  > Limited admin practice isn’t just to save money. It’s a security practice and it’s a good one. And it's a red flag. Means that IT is running the show, not engineering.
- spondyl 5y ago
  I agree in principle! To be a bit clearer, I was thinking more in terms of locality so for example, "Users don't even have access to their laptops right in front of them" let alone visibility into their own department or even wider organisation due to access policy restrictions. Being able to write or edit everywhere would be ridiculous but sometimes there are (admittedly intangible) upsides to transparency (read only access in a technical sense) to most aspects of the wider business
- apetresc 5y ago
  You think the _legal department_ understands the whole business better than anyone? Where have you been working?
  - aynyc 5y ago
    Every company I work for, legal usually sign of every deal, in terms of legality of the contract. They know every contracts/clients we have. They know where all our business is. They might not care whether our business plan is or works, but they know what our business is. Same can be say about finance group.
908B64B197 5y ago
Also, the caliber of engineers at Google might not be the same as RandomCo where admin access isn't allowed for engineers.

tyingq 5y ago

"You know what? We need more layers. On top of this." Heh. Service Mesh!

rantwasp 5y ago
K8s on K8s! K64s!
- voidfunc 5y ago
  But can it run on an N64?
  - rantwasp 5y ago
    it starts at Nintendo 69.
- yongjik 5y ago
  Kuberneteuberneteuberneteuberneteuberneteuberneteuberneteubernetes is a perfect name for the beautiful system you are about to architect: it's an ancient Greek word for helmsman-man-man-man-man-man-man-man, who is in charge of helmsman-man-man-man-man-man-man, who is in charge of ... (...sorry =_=)
- verst 5y ago
  You joke, but have you looked at K8s VirtualCluster? Separate control planes for each tenant within a K8s cluster. https://github.com/kubernetes-sigs/cluster-api-provider-nested/tree/main/virtualcluster https://github.com/kubernetes-sigs/cluster-api-provider-nest...
  - rantwasp 5y ago
    of course. of course! it’s k8s clusters all the way down
- lubujackson 5y ago
  "Nobody will ever need more than K64." - Robo-Gates

tene 5y ago

I'm in kind of the opposite position, as I feel confused about why I keep seeing so much resistance to higher levels of abstraction here. > It boggles my mind that people look at something like Kubernetes and decide "You know what? We need more layers. On top of this." I'm not involved with this project at all, but yes, I look at the massive Kubernetes deployments I've worked with, and conclude that toil and various other kinds of problems could be reduced with higher-level abstractions for declaratively managing configuration for all of these clusters. If you wanted to run workloads on 100k servers, how would you do it? Would you have a team manually configure each cluster individually, or would you consider that there are a lot of similarities between clusters, and you only want differences that are chosen intentionally, and look into some abstraction to help keep the complexity down? If 100k isn't enough for you, is there any number of nodes at which you'd consider building some tools support for your work? What about data center count and change rate? There are a lot of business models that benefit from running workloads closer to customers, so it's useful to have deployments in many datacenters and clouds across the world, and for this to grow over time. Do you really not see why some people are interested in being able to bring up a cluster in a new DC by updating a few configs rather than starting from nothing by hand with kubeadm init? I apologize if I've misrepresented your position by misunderstanding something. I feel like I'm missing or misunderstanding something from your perspective, but I don't know what it is. More generally, I think it's extremely normal for people to build higher levels of abstraction as things grow. It's usually worth considering abstraction or architectural changes in a system every time it grows by about 10x. Some of the most painful platforms I've supported in my career have been those that didn't do this, and instead tried to grow through brute force. Something can be fine to do once, troublesome to do 10 times, and overwhelming to do 100 times.

smichel17 5y ago
I think the comment you are replying to was objecting to the number of layers, rather than the level, of abstraction. More concretely, I'd guess they would not object to a tool that replaces k8s and offers a higher level api, but they do object to building said tool on top of k8s. For myself, I think it depends on how leaky the abstractions are. The leakier, the fewer layers the house of cards can stand. That is, it's more about where you slice it than how many times.

Groxx 5y ago

With a sufficiently-large N, N+1 is barely noticeable.

throwaway894345 5y ago

> Ostensibly, it's so that you can isolate those fucking jobs so that different teams can run different tasks in the same cluster without interfering with each other. I think the point is so the teams can run services at all. Isolation isn’t the point, it’s breaking a dependency on an ops team (Kubernetes in a sense is ops as a service).

jldugger 5y ago

> Fire up another business until capital runs out, and if it does, no worries, jut hop to another job! I hear people say Google runs their business this way. So perhaps it makes sense that Kubernetes came from them.

sangnoir 5y ago

> At this point, why not just drive it up to the logical conclusion? At this point, why not just drive it up to the logical conclusion? Treat your business model as cattle, not pets. We're already there. Cue "We invest in teams not ideas"[1,2] and "We're pivoting since we couldn't get a product-market match, but have millions in the bank"[3] 1.https://www.forbes.com/sites/ilyapozin/2015/10/08/4-reasons-to-invest-in-people-not-companies/?sh=4c8fd6bc47cb https://www.forbes.com/sites/ilyapozin/2015/10/08/4-reasons-... 2. https://www.gsb.stanford.edu/insights/do-funders-care-more-about-your-team-your-idea-or-your-passion https://www.gsb.stanford.edu/insights/do-funders-care-more-a... 3. https://triplebyte.com/blog/rethinking-triplebyte https://triplebyte.com/blog/rethinking-triplebyte

dexterous 5y ago

The only explanation I have that makes any sense is that it is a very Application Developer solution to a System Administration that hasn't matured it's abstractions and is now trapped in Tech Debt purgatory.

bradhe 5y ago

> It boggles my mind that people look at something like Kubernetes and decide "You know what? We need more layers. On top of this." This statement is less about adding another layer. It's more about not falling in to the same trap that we did with VMs in 2010--managing them by hand, producing unreproducible configurations, "tending" to them. The same pattern is happening with Kubernetes, but it fundamentally runs on computers at the end of the day so it's prone to similar types of failures.