29 ms·

Kubernetes: The Documentary

263 points by wallflower 5y ago

tester756 5y ago
Why Google despite having Borg and know-how so early wasn't 1st to the Cloud? or bigger threat to AWS/Azure?
- jthrowsitaway 5y ago
  Google's advertising business is really profitable so anything else is secondary. Amazon came at it from a business angle. They had a ton of spare resources that they only needed once in a while and figured out a way to make use of them.
  - randomsilence 5y ago
    >They had a ton of spare resources Is this true? There was the story about the resources they need for Christmas shopping that went unused for the rest of the year. But recently somebody mentioned here (I don't remember where) that AWS was a separate business and that Amazon itself initially didn't use any resources from AWS.
    - FranGro78 5y ago
      That’s correct, the multi year project was known internally as MAWS - Move to AWS.
    - dastbe 5y ago
      building a public cloud is more or less a rebuild of any infrastructure you currently have. it would be just unwise for a company to build their private cloud with the same trade offs as public cloud. so while there’s the ideal state of running amazon on a large multi tenant cloud, there was absolutely no safe and reasonable path to making that happen by extending the existing infra. it would’ve also hampered aws by forcing them to focus way too large way too early.
  - pid-1 5y ago
    amazon.com also was really profitable when AWS was launched. IDK, at this point I think we can just agree Google is bad at diversifying. The day their ad business dies, they're done.
    - kortilla 5y ago
      It was profitable, but nothing like Google’s margins. It was still just a glorified Walmart with delivery (i.e. not much of a moat).
    - emodendroket 5y ago
      The fear that they were plateauing was a major motivator. Google doesn’t really face that.
    - oblio 5y ago
      That could take a ton of time to die off and they could still pivot. But they're the least diversified, after Facebook, agreed.
    - ec109685 5y ago
      Amazon.com wasn’t really profitable. They ran at low margins for years. In 2006, their profit was like 500m against 10B in revenue.
      - jethro_tell 5y ago
        Because they were using all profit to build out EC2, fulfilment centers and fullfilment tech. It wasn't that there wasn't profit because the margins were low, there wasn't profit because they were pouring it all into different verticals and building moats. So yeah they weren't profitable but they weren't just scraping by either. I think that's one of those things that people miss when they compare a not-so-profitable startup to Amazon. The size, scale, and huge number of product markets that Amazon plays in is really pretty astounding for a company that is only 25 years old.
  - nevir 5y ago
    > They had a ton of spare resources that they only needed once in a while and figured out a way to make use of them. Not actually the case. AWS was a completely independent stack (from hardware on up) - we couldn't use it internally for quite some time (years); and even then, only in a limited fashion (e.g. only S3 for some time)
  - jeffbee 5y ago
    Bingo. There are two facts you need to understand Google's relationship with GCP. Cloud margins are the worst margins of anything at Google, so if there has to be a choice on where to spend money, it can only go to Cloud begrudgingly, or as part of some long game. Second thing you need to know is Google's internal workloads are way, way bigger than you are imagining. So if you see a headline that Google invested $100 billion in datacenters, don't imagine that all, most, or even a lot of that went toward GCP capacity. They spend that money on search, advertising, and letting an AI play ten million years of StarCraft.
- mohanmcgeek 5y ago
  They didn't realise it was a high margin business until AWS numbers were reported separately. A lot of people used to think that Amazon.com was subsidising AWS. Financials from other VPS providers suggested this to be the case.
  - ec109685 5y ago
    GCP has been in the cloud business since 2008, two years after aws launched. They clearly were in a position to predict AWS’s profit margins before AWS publicized them given they were competing for same customers.
    - jakupovic 5y ago
      I remember when GCP launched, we had an all hands meeting for all of AWS, which all fit in a big conference room, less than 100 people. People thought now that Google got in the business that was it. The leadership, Andy Jassy, assured everyone that our jobs were safe and we could get back to work so to say. Seems funny now, but at the time AWS was S3 and EC2 and google was ahead it seemed.
      - mohanmcgeek 5y ago
        When was this? 2012?
        jakupovic 5y ago
        2008 when GCP launched.
    - mohanmcgeek 5y ago
      Compute Engine was announced at Google I/O 2012. GA in 2013 Google app engine was definitely NOT competing with AWS. They were not selling to enterprises.
- sorry_outta_gas 5y ago
  Google is poor at support, web hosting/cloud is as much about support as it is technology
  - hackandtrip 5y ago
    In my little experience, although GCP docs leaves a lot to be desired, the support was always very thorough and fast to answer, even for not critical/emergency situations.
    - sorry_outta_gas 5y ago
      They have improved a lot recently
    - pm90 5y ago
      One of the frustrating thing with their docs is that they keep changing them without versioning (last time I checked). So it’s easy to feel like you’re being gaslighted because assumptions you made on the basis of the docs could have changed and then you look stupid. AWS being the dominant player means that if they make any silent changes there will be a SO/GitHub issue/HN thread about it. But with GCP without their pricy professional tier support you can feel a bit unsure and lost all the time about your understanding of it.
  - paulluuk 5y ago
    To be honest, AWS cloud support isn't that fantastic either. Responses to tickets usually come in 48 hours, if not more. I don't know about you, but in 48 hours I've usually already built a workaround and moved on to other projects. Not saying that AWS support is worse or even equal to GCP support, but just commenting that cloud support in general feels pretty useless and an afterthought at best. And let's not even get started on the horrible, awful UI in AWS console.
    - grogenaut 5y ago
      pay for enterprise support, response is within minutes
- emodendroket 5y ago
  One theory I’ve heard (I think from Yegge?) is that Google’s internal tooling is so good at dealing with breaking changes that they’re too cavalier about introducing them.
  - dastbe 5y ago
    if you have the right company setup, headcount, and culture then you can absolutely build a world where teams can be allowed to force “perfection” over time, even with amazing levels of automation. i think the interesting thing is that while you’d think that is the ideal everyone should strive for, the reality is that’s inapplicable or detrimental to run most businesses that way.
    - oblio 5y ago
      > build a world where teams can be allowed to force “perfection” over time, even with amazing levels of automation. I'm confused, did you mean "without" instead of "with"? Also: > i think the interesting thing is that while you’d think that is the ideal everyone should strive for, the reality is that’s inapplicable or detrimental to run most businesses that way. Why would you say that is?
      - dastbe 5y ago
        i meant with. google has built some really great tools for minimizing toil when pushing major changes across a codebase. even then, there’s a nontrivial overhead that is borne across the company. > Why would you say that is? many/most businesses don’t have the margins/scale/time to focus on it. in a lot of businesses (amazons included) the goal for the business is to try as many things as fast as possible and scale that which works and abandon what fails. the goal of infrastructure there is to get out of the way and enable the layers above to build things as naively and simply as possible with little to no churn.
      - emodendroket 5y ago
        I think we see a good reason here: normal people don't think "this API is so beautiful," they think "these clowns are making me do a migration yet again."
- singron 5y ago
  Also, Google's infrastructure is so vertically integrated that it's not obvious how to offer it others. E.g. Borg assigns 1 IP to each machine, which means every pod-equivalent has to listen on a dynamically allocated port, so Google eschewed DNS in favor of a system that more easily assigned dynamic IP:port pairs without relying on TTLs (BNS/Chubby). Google also decoupled storage from compute and had a bunch of custom databases to run in that paradigm (GFS/CFS/BigTable).
  - deanCommie 5y ago
    The mistake also reveals the constrained imagination: Google could only conceive of offering what they had internally to others, struggled, and continues to struggle. Amazon didn't try - they just knew that they were good at building high scale distributed systems, and thought they could build services for others. That's what S3 and SQS and EC2 were - built from scratch, not trying to figure out how to externalize the internals.
    - ec109685 5y ago
      GCP was built from scratch as well.
      - singron 5y ago
        It's more complicated, but I think everything is at least a substantial layer on top of an internal offering. E.g. GCE runs VMs in Borg using the same physical infrastructure used for other services, but GCE has its own API and virtualized networking on top that hides Borg and was not (at the time) used for any internal services.
  - ra7 5y ago
    I’ve heard Google considers (considered?) their infrastructure prowess as a secret sauce. Could it be that they didn’t want anyone else to benefit from it? They had years of headstart on most infrastructure technology that’s ubiquitous now, but didn’t bother to offer it to others. Perhaps they thought only Google should benefit from it, while Amazon saw a business opportunity with AWS.
- deanCommie 5y ago
  Hubris. Steve Yegge talks about it on his podcast: https://youtu.be/9v4z46Ea35Q https://youtu.be/9v4z46Ea35Q Amazon actually works backwards like they say - they ask customers what they want, and build it. Google thinks they're smarter than everyone else so they can't comprehend that developers out there in the world want different things than the developers inside Google. You can see this permeate even in this documentary, which is ostensibly about a success - when Google actually did launch something that developers really liked - K8S. They didn't start from developer needs or wants. They started by noticing AWS was eating their lunch, and wanting to compete with them, but realizing they couldn't win by doing it directly.
  - ec109685 5y ago
    To be fair, K8s has caught on even outside of GCP. Also, while Amazon does work backwards from customers, part of them working backwards is how to translate customer requirements into a successful business. Amazon is doing this as a for profit company, not spending money on altruistic initiatives. E.g. purely for the benefit of customers, having AWS services be multi-cloud and open source or reducing egress cost would be best.
    - p_l 5y ago
      Because K8s, while commonly deployed on the cloud, is more of a "local cloud operating system" that you use to marshall a bunch of resources into "cloud" (and finalizing TPRs into CRDs really made it shine). To the point that k8s is base operating system for... Chick-Fil-A restaurants, deployed locally, controlling the industrial kitchen hardware.
- ec109685 5y ago
  Borg was good enough that google engineers didn’t have any desire to use GCP and thus most of the company was not dogfooding its own product. Additionally, the “two pizza teams” style of Amazon’s development was more amenable to the AWS model of development where each piece of Amazon could run effectively in AWS [1]. Google is structured differently, and it’s only recently that both AWS and GCP could even approach running the largest systems from a company like Google. [1] Yes I know that AWS wasn’t built specifically for Amazon engineers, but the fact that it could accommodate them over time has been a boon for their cloud, and it gives enterprises confidence that of AWS’s strategic importance and desire to make it world class. There isn’t a large hidden layer of services that Amazon has access to that the general public doesn’t.
- servytor 5y ago
  It is because Amazon dog-fooded their own technology stack. This forced them to have a better interface to their infrastructure, which in turn made it easy to open to the public. Google on the other hand, hid behind their own complexity and let engineers feel stupid for not knowing Borg like the back of their hand.
- debuggerpk 5y ago
  I think Google did .. App Engine came around 2008.
baalimago 5y ago
Who is this documentary for? Anyone who's interested surely doesn't need an explanation on what a docker container, or open source software, is
- hackandtrip 5y ago
  I am no expert, but I know what a container is and I use it in my daily job; same for Kube, but hearing the actual people going through it feels a bit like an history lesson. (The documentary also touches various interesting point, such as open-source in the eyes of upper management, and so on... surely there is some better source/book/article, but I found the format pretty chill and easy to absorb)
- epgui 5y ago
  I think this is high-level enough to appeal to non-technical people. I've shared this with some friends who have money invested in cloud companies they don't really understand, and with my family for whom even my simplified job description is too abstract to really comprehend. I think this is good, but it makes me wish for an even broader-scoped documentary on the history of cloud technologies.
  - pm90 5y ago
    > I think this is good, but it makes me wish for an even broader-scoped documentary on the history of cloud technologies. Honestly I would love for there to be more documentaries on the more “boring” computing related stuff too, like what are the origins of VMs. There’s a lot of pop culture material on the founding of the big tech but very little on the boring stuff without which none of this would have been possible. And my assumptions about who invented what when are always changed by talking with people who have been around for long enough. If there were a dedicated field of Academic History dedicated to computing that would be amazing. Just having a timeline that explains the lineage of computer technologies without it being a short paragraphs in the introduction chapter in books on more specific tools, that would be great.
    - scott_s 5y ago
tombh 5y ago
I love Kubernetes, but this documentary makes me sad. The story of PaaS is absolutely worthy of a documentary, but to imply that Google is the protagonist is just pandering to the "rich kid" who brought little more to the table than money and influence. Here's a better sketch for the story. It starts with Ruby, itself a game-changing language, that inspired a new culture of developer-centric best practices like Rspec's TDD and Bundler's lockfiles. Ruby made the first PaaS possible, Heroku (a YC startup), which itself inspired a whole new way of thinking about software. I don't think enough people realise just how influential Heroku's 12 Factor App manifesto[1] was, in fact I'd argue it's the real hero in this whole story. The 12 Factors are just text, it's not even software or hard and fast requirements, they're merely best practices for the entire experience of writing, collaborating, testing, deploying, scaling and maintaining software. The 12 Factors aren't just about pushing your app on Kubernetes or whatever, they're just as much, if not more so, ideas that make _you_ a better engineer. Heroku made money for sure, but they did something beyond money, they upped the level of our craft as a whole. The central concept in Heroku is the dyno. If you follow the 12 Factors, then you magically get almost infinitely scalable dynos. It plays right into the heart of the new gold rush world of startups, where anybody can have an app idea and leverage the global marketplace with the abstractions of datacentres and the cloud. Docker is the open source dyno. So many PaaS projects apart from Kubernetes have risen and fallen around Docker. Deis, Flynn, Dokku, Swarm, to name but a few. So many of the core concepts of Kubernetes were innovated and explored, and arguably mastered, before Kubernetes came on the scene. Kubernetes is a success not because of its innovation, but because it's backed by The Rich Kid. The world was already heading in this direction, Google just threw big numbers at it to make sure they took the biggest piece of the pie. 1. https://12factor.net https://12factor.net
- xcambar 5y ago
  I've been involved in multiple migrations to public cloud infrastructure (some were unnecessary, but such are the trends...), and without failure, the 12 factors manifesto shows up in the very first days of the discussions as the first target to reach before anything else is doable. I very much enjoy it and I'm glad whether I see someone mention it or I can spread and teach its wisdom.
- 5y ago
throwawaymanbot 5y ago
coding123 5y ago
God even their aspect ratio has to be pretentious.
siliconc0w 5y ago
In some respects K8s is a bit of a step backwards from Heroku or AppEngine. It's complicated and requires you master its learning curves to deliver benefits. This looks like a lot of cloud products which are becoming more complicated to sell to bigger companies where they have the ops teams dedicated learning these products. You get an 'easy' to setup demo with a lot of copy-paste or bash|sh but you'll need a PhD for running your own apps in production. And each product is a different learning curve and they don't always play nice. The next paradigm is composability and simplification. I should be able to define an application, its configuration, what it needs to talk to, and level of desired spend for which availability/performance SLO and it should mostly just work. Security should just work and each perimeter is automatically configured - without managing separate configuration for admin APIs, end user auth, peer auth, k8 network policies, firewall policies, etc. I shouldn't have to care about runtime - GKE vs Cloud Run vs Lambda or optimizing Node shapes or whatever new product launched yesterday. If I push a production change there should be a consistent way of simulating, canarying, and slowing rolling that change out across all products. Ideally, it should basically be a marketplace of commodity network/compute/storage and the providers are dynamically bidding for my business. If you can run my application as fast or as reliable with less spend, you get the business - if not the app magically moves the provider that can.
- odonnellryan 5y ago
  My problem with kubernetes is for a lot of stuff you don't know what is happening really. I recently installed cert manager and the config file alone for that resource was 5k lines generated by helm. That is just crazy. I want to know generally how that part of the app is configured (it is a user facing cert) and it isn't that complicated in the real world when you set things up with nginx. I feel like I probably should trust the people working on these products more than I trust me securing a Linux box, but... I dunno if I'm there yet.
- rektide 5y ago
  > In some respects K8s is a bit of a step backwards from Heroku or AppEngine. It's complicated and requires you master its learning curves to deliver benefits. what, exactly, do you think kubernetes is? what is the central design philosohy of it? how does it work? can you name anything about kubernetes that makes it better or different than Heroku or AppEngine? nothing you've said gives me confidenc you can identify the most basic strengths or competencies here, that you can identify how kubernetes is different from the more prosaic & narrow platforms which came before it &nehat that meams. kis is a different take than just an app delivery platform, but i dont see that recognizance here. to me, you are propoing we should all switch from planes to bicycles because bicycles are simpler & all folks need. this reductionism is a misservice to real debate over what & where we should go. endless consumerization of computing, treating all computing workloads as services for petty end-users, who need dumb simple integrated ways to continue shipping legacy software platforms, bundled into fancier containers, is a dead end shithole future. by themselves, simplicity & composability buy us nothing but low-ended-ness. they reduce our ability to think & integrate & fins better cloud patterns to make software with. thankfully i see little potential threat out there from this proposal that we make ourselves more naive. tbis beleif that we can just somehow snap our fingers & simplify & the complexity goes away doesnt seem to ne enough. what i do have faith in are better management tools & practices taht emerge atop & integrating with existing full-blooded clous platforma like kunernetes. folks like ArgoCD or Flux provide a pretty reasonable seveloper experience somewhere reasonably between Herokus bramch-based deployment model & the modern open source cloud platform. last, but definitely not least: what specifically does your comment have to do with this link that was shared? i dont see any direct connection to the link. it seems like you just wanted to dump a generic opinion on us, to neg kubernetes broadly. how if at all have you engaged the material in thia video in your negging? edit: very quickly to -2. im not playing kind but i think silicon deserved heavily to be challenged. if you disagree, i think words are appropriate.
crb 5y ago
If you want to learn the history of Kubernetes from the people that made it, I host a weekly Kubernetes Podcast. If you watched this documentary, you're the target audience. (If you're interested in the docume Many of the people in this documentary have been on the show before; I recommend episodes with Brian Grant, Dawn Chen, Tim Hockin and Clayton Coleman. https://kubernetespodcast.com/ https://kubernetespodcast.com/
- pm90 5y ago
  +1, great podcast. I used to listen to this one regularly. But the amount of stuff they talked about was just too great to track. So now I listen to specific topics when I may need to use in order to get a informal overview.
deleted 5y ago
[deleted]
hahamaster 5y ago
Oh I was so important, me too, I am very important and we were visionaries solving very important problems that impact millions. It turned out it was very important but it didn't seem that way in retrospect. I'm also very important but also work very well with buzzwords in a way that says absolutely nothing, except that I'm important but we totally didn't know it at the time. None of us wrote a line of code but we were nonetheless very important, and, more importantly, we're even more important now.

263 points by wallflower 5y ago

tester756 5y ago

Why Google despite having Borg and know-how so early wasn't 1st to the Cloud? or bigger threat to AWS/Azure?

jthrowsitaway 5y ago
Google's advertising business is really profitable so anything else is secondary. Amazon came at it from a business angle. They had a ton of spare resources that they only needed once in a while and figured out a way to make use of them.
- randomsilence 5y ago
  >They had a ton of spare resources Is this true? There was the story about the resources they need for Christmas shopping that went unused for the rest of the year. But recently somebody mentioned here (I don't remember where) that AWS was a separate business and that Amazon itself initially didn't use any resources from AWS.
  - FranGro78 5y ago
    That’s correct, the multi year project was known internally as MAWS - Move to AWS.
  - dastbe 5y ago
    building a public cloud is more or less a rebuild of any infrastructure you currently have. it would be just unwise for a company to build their private cloud with the same trade offs as public cloud. so while there’s the ideal state of running amazon on a large multi tenant cloud, there was absolutely no safe and reasonable path to making that happen by extending the existing infra. it would’ve also hampered aws by forcing them to focus way too large way too early.
- pid-1 5y ago
  amazon.com also was really profitable when AWS was launched. IDK, at this point I think we can just agree Google is bad at diversifying. The day their ad business dies, they're done.
  - kortilla 5y ago
    It was profitable, but nothing like Google’s margins. It was still just a glorified Walmart with delivery (i.e. not much of a moat).
  - emodendroket 5y ago
    The fear that they were plateauing was a major motivator. Google doesn’t really face that.
  - oblio 5y ago
    That could take a ton of time to die off and they could still pivot. But they're the least diversified, after Facebook, agreed.
  - ec109685 5y ago
    Amazon.com wasn’t really profitable. They ran at low margins for years. In 2006, their profit was like 500m against 10B in revenue.
    - jethro_tell 5y ago
      Because they were using all profit to build out EC2, fulfilment centers and fullfilment tech. It wasn't that there wasn't profit because the margins were low, there wasn't profit because they were pouring it all into different verticals and building moats. So yeah they weren't profitable but they weren't just scraping by either. I think that's one of those things that people miss when they compare a not-so-profitable startup to Amazon. The size, scale, and huge number of product markets that Amazon plays in is really pretty astounding for a company that is only 25 years old.
- nevir 5y ago
  > They had a ton of spare resources that they only needed once in a while and figured out a way to make use of them. Not actually the case. AWS was a completely independent stack (from hardware on up) - we couldn't use it internally for quite some time (years); and even then, only in a limited fashion (e.g. only S3 for some time)
- jeffbee 5y ago
  Bingo. There are two facts you need to understand Google's relationship with GCP. Cloud margins are the worst margins of anything at Google, so if there has to be a choice on where to spend money, it can only go to Cloud begrudgingly, or as part of some long game. Second thing you need to know is Google's internal workloads are way, way bigger than you are imagining. So if you see a headline that Google invested $100 billion in datacenters, don't imagine that all, most, or even a lot of that went toward GCP capacity. They spend that money on search, advertising, and letting an AI play ten million years of StarCraft.
mohanmcgeek 5y ago
They didn't realise it was a high margin business until AWS numbers were reported separately. A lot of people used to think that Amazon.com was subsidising AWS. Financials from other VPS providers suggested this to be the case.
- ec109685 5y ago
  GCP has been in the cloud business since 2008, two years after aws launched. They clearly were in a position to predict AWS’s profit margins before AWS publicized them given they were competing for same customers.
  - jakupovic 5y ago
    I remember when GCP launched, we had an all hands meeting for all of AWS, which all fit in a big conference room, less than 100 people. People thought now that Google got in the business that was it. The leadership, Andy Jassy, assured everyone that our jobs were safe and we could get back to work so to say. Seems funny now, but at the time AWS was S3 and EC2 and google was ahead it seemed.
    - mohanmcgeek 5y ago
      When was this? 2012?
      - jakupovic 5y ago
        2008 when GCP launched.
  - mohanmcgeek 5y ago
    Compute Engine was announced at Google I/O 2012. GA in 2013 Google app engine was definitely NOT competing with AWS. They were not selling to enterprises.
sorry_outta_gas 5y ago
Google is poor at support, web hosting/cloud is as much about support as it is technology
- hackandtrip 5y ago
  In my little experience, although GCP docs leaves a lot to be desired, the support was always very thorough and fast to answer, even for not critical/emergency situations.
  - sorry_outta_gas 5y ago
    They have improved a lot recently
  - pm90 5y ago
    One of the frustrating thing with their docs is that they keep changing them without versioning (last time I checked). So it’s easy to feel like you’re being gaslighted because assumptions you made on the basis of the docs could have changed and then you look stupid. AWS being the dominant player means that if they make any silent changes there will be a SO/GitHub issue/HN thread about it. But with GCP without their pricy professional tier support you can feel a bit unsure and lost all the time about your understanding of it.
- paulluuk 5y ago
  To be honest, AWS cloud support isn't that fantastic either. Responses to tickets usually come in 48 hours, if not more. I don't know about you, but in 48 hours I've usually already built a workaround and moved on to other projects. Not saying that AWS support is worse or even equal to GCP support, but just commenting that cloud support in general feels pretty useless and an afterthought at best. And let's not even get started on the horrible, awful UI in AWS console.
  - grogenaut 5y ago
    pay for enterprise support, response is within minutes
emodendroket 5y ago
One theory I’ve heard (I think from Yegge?) is that Google’s internal tooling is so good at dealing with breaking changes that they’re too cavalier about introducing them.
- dastbe 5y ago
  if you have the right company setup, headcount, and culture then you can absolutely build a world where teams can be allowed to force “perfection” over time, even with amazing levels of automation. i think the interesting thing is that while you’d think that is the ideal everyone should strive for, the reality is that’s inapplicable or detrimental to run most businesses that way.
  - oblio 5y ago
    > build a world where teams can be allowed to force “perfection” over time, even with amazing levels of automation. I'm confused, did you mean "without" instead of "with"? Also: > i think the interesting thing is that while you’d think that is the ideal everyone should strive for, the reality is that’s inapplicable or detrimental to run most businesses that way. Why would you say that is?
    - dastbe 5y ago
      i meant with. google has built some really great tools for minimizing toil when pushing major changes across a codebase. even then, there’s a nontrivial overhead that is borne across the company. > Why would you say that is? many/most businesses don’t have the margins/scale/time to focus on it. in a lot of businesses (amazons included) the goal for the business is to try as many things as fast as possible and scale that which works and abandon what fails. the goal of infrastructure there is to get out of the way and enable the layers above to build things as naively and simply as possible with little to no churn.
    - emodendroket 5y ago
      I think we see a good reason here: normal people don't think "this API is so beautiful," they think "these clowns are making me do a migration yet again."
singron 5y ago
Also, Google's infrastructure is so vertically integrated that it's not obvious how to offer it others. E.g. Borg assigns 1 IP to each machine, which means every pod-equivalent has to listen on a dynamically allocated port, so Google eschewed DNS in favor of a system that more easily assigned dynamic IP:port pairs without relying on TTLs (BNS/Chubby). Google also decoupled storage from compute and had a bunch of custom databases to run in that paradigm (GFS/CFS/BigTable).
- deanCommie 5y ago
  The mistake also reveals the constrained imagination: Google could only conceive of offering what they had internally to others, struggled, and continues to struggle. Amazon didn't try - they just knew that they were good at building high scale distributed systems, and thought they could build services for others. That's what S3 and SQS and EC2 were - built from scratch, not trying to figure out how to externalize the internals.
  - ec109685 5y ago
    GCP was built from scratch as well.
    - singron 5y ago
      It's more complicated, but I think everything is at least a substantial layer on top of an internal offering. E.g. GCE runs VMs in Borg using the same physical infrastructure used for other services, but GCE has its own API and virtualized networking on top that hides Borg and was not (at the time) used for any internal services.
- ra7 5y ago
  I’ve heard Google considers (considered?) their infrastructure prowess as a secret sauce. Could it be that they didn’t want anyone else to benefit from it? They had years of headstart on most infrastructure technology that’s ubiquitous now, but didn’t bother to offer it to others. Perhaps they thought only Google should benefit from it, while Amazon saw a business opportunity with AWS.
deanCommie 5y ago
Hubris. Steve Yegge talks about it on his podcast: https://youtu.be/9v4z46Ea35Q https://youtu.be/9v4z46Ea35Q Amazon actually works backwards like they say - they ask customers what they want, and build it. Google thinks they're smarter than everyone else so they can't comprehend that developers out there in the world want different things than the developers inside Google. You can see this permeate even in this documentary, which is ostensibly about a success - when Google actually did launch something that developers really liked - K8S. They didn't start from developer needs or wants. They started by noticing AWS was eating their lunch, and wanting to compete with them, but realizing they couldn't win by doing it directly.
- ec109685 5y ago
  To be fair, K8s has caught on even outside of GCP. Also, while Amazon does work backwards from customers, part of them working backwards is how to translate customer requirements into a successful business. Amazon is doing this as a for profit company, not spending money on altruistic initiatives. E.g. purely for the benefit of customers, having AWS services be multi-cloud and open source or reducing egress cost would be best.
  - p_l 5y ago
    Because K8s, while commonly deployed on the cloud, is more of a "local cloud operating system" that you use to marshall a bunch of resources into "cloud" (and finalizing TPRs into CRDs really made it shine). To the point that k8s is base operating system for... Chick-Fil-A restaurants, deployed locally, controlling the industrial kitchen hardware.
ec109685 5y ago
Borg was good enough that google engineers didn’t have any desire to use GCP and thus most of the company was not dogfooding its own product. Additionally, the “two pizza teams” style of Amazon’s development was more amenable to the AWS model of development where each piece of Amazon could run effectively in AWS [1]. Google is structured differently, and it’s only recently that both AWS and GCP could even approach running the largest systems from a company like Google. [1] Yes I know that AWS wasn’t built specifically for Amazon engineers, but the fact that it could accommodate them over time has been a boon for their cloud, and it gives enterprises confidence that of AWS’s strategic importance and desire to make it world class. There isn’t a large hidden layer of services that Amazon has access to that the general public doesn’t.
servytor 5y ago
It is because Amazon dog-fooded their own technology stack. This forced them to have a better interface to their infrastructure, which in turn made it easy to open to the public. Google on the other hand, hid behind their own complexity and let engineers feel stupid for not knowing Borg like the back of their hand.
debuggerpk 5y ago
I think Google did .. App Engine came around 2008.

baalimago 5y ago

Who is this documentary for? Anyone who's interested surely doesn't need an explanation on what a docker container, or open source software, is

hackandtrip 5y ago
I am no expert, but I know what a container is and I use it in my daily job; same for Kube, but hearing the actual people going through it feels a bit like an history lesson. (The documentary also touches various interesting point, such as open-source in the eyes of upper management, and so on... surely there is some better source/book/article, but I found the format pretty chill and easy to absorb)
epgui 5y ago
I think this is high-level enough to appeal to non-technical people. I've shared this with some friends who have money invested in cloud companies they don't really understand, and with my family for whom even my simplified job description is too abstract to really comprehend. I think this is good, but it makes me wish for an even broader-scoped documentary on the history of cloud technologies.
- pm90 5y ago
  > I think this is good, but it makes me wish for an even broader-scoped documentary on the history of cloud technologies. Honestly I would love for there to be more documentaries on the more “boring” computing related stuff too, like what are the origins of VMs. There’s a lot of pop culture material on the founding of the big tech but very little on the boring stuff without which none of this would have been possible. And my assumptions about who invented what when are always changed by talking with people who have been around for long enough. If there were a dedicated field of Academic History dedicated to computing that would be amazing. Just having a timeline that explains the lineage of computer technologies without it being a short paragraphs in the introduction chapter in books on more specific tools, that would be great.
  - scott_s 5y ago

tombh 5y ago

I love Kubernetes, but this documentary makes me sad. The story of PaaS is absolutely worthy of a documentary, but to imply that Google is the protagonist is just pandering to the "rich kid" who brought little more to the table than money and influence. Here's a better sketch for the story. It starts with Ruby, itself a game-changing language, that inspired a new culture of developer-centric best practices like Rspec's TDD and Bundler's lockfiles. Ruby made the first PaaS possible, Heroku (a YC startup), which itself inspired a whole new way of thinking about software. I don't think enough people realise just how influential Heroku's 12 Factor App manifesto[1] was, in fact I'd argue it's the real hero in this whole story. The 12 Factors are just text, it's not even software or hard and fast requirements, they're merely best practices for the entire experience of writing, collaborating, testing, deploying, scaling and maintaining software. The 12 Factors aren't just about pushing your app on Kubernetes or whatever, they're just as much, if not more so, ideas that make _you_ a better engineer. Heroku made money for sure, but they did something beyond money, they upped the level of our craft as a whole. The central concept in Heroku is the dyno. If you follow the 12 Factors, then you magically get almost infinitely scalable dynos. It plays right into the heart of the new gold rush world of startups, where anybody can have an app idea and leverage the global marketplace with the abstractions of datacentres and the cloud. Docker is the open source dyno. So many PaaS projects apart from Kubernetes have risen and fallen around Docker. Deis, Flynn, Dokku, Swarm, to name but a few. So many of the core concepts of Kubernetes were innovated and explored, and arguably mastered, before Kubernetes came on the scene. Kubernetes is a success not because of its innovation, but because it's backed by The Rich Kid. The world was already heading in this direction, Google just threw big numbers at it to make sure they took the biggest piece of the pie. 1. https://12factor.net https://12factor.net

xcambar 5y ago
I've been involved in multiple migrations to public cloud infrastructure (some were unnecessary, but such are the trends...), and without failure, the 12 factors manifesto shows up in the very first days of the discussions as the first target to reach before anything else is doable. I very much enjoy it and I'm glad whether I see someone mention it or I can spread and teach its wisdom.
5y ago

throwawaymanbot 5y ago

coding123 5y ago

God even their aspect ratio has to be pretentious.

siliconc0w 5y ago

In some respects K8s is a bit of a step backwards from Heroku or AppEngine. It's complicated and requires you master its learning curves to deliver benefits. This looks like a lot of cloud products which are becoming more complicated to sell to bigger companies where they have the ops teams dedicated learning these products. You get an 'easy' to setup demo with a lot of copy-paste or bash|sh but you'll need a PhD for running your own apps in production. And each product is a different learning curve and they don't always play nice. The next paradigm is composability and simplification. I should be able to define an application, its configuration, what it needs to talk to, and level of desired spend for which availability/performance SLO and it should mostly just work. Security should just work and each perimeter is automatically configured - without managing separate configuration for admin APIs, end user auth, peer auth, k8 network policies, firewall policies, etc. I shouldn't have to care about runtime - GKE vs Cloud Run vs Lambda or optimizing Node shapes or whatever new product launched yesterday. If I push a production change there should be a consistent way of simulating, canarying, and slowing rolling that change out across all products. Ideally, it should basically be a marketplace of commodity network/compute/storage and the providers are dynamically bidding for my business. If you can run my application as fast or as reliable with less spend, you get the business - if not the app magically moves the provider that can.

odonnellryan 5y ago
My problem with kubernetes is for a lot of stuff you don't know what is happening really. I recently installed cert manager and the config file alone for that resource was 5k lines generated by helm. That is just crazy. I want to know generally how that part of the app is configured (it is a user facing cert) and it isn't that complicated in the real world when you set things up with nginx. I feel like I probably should trust the people working on these products more than I trust me securing a Linux box, but... I dunno if I'm there yet.
rektide 5y ago
> In some respects K8s is a bit of a step backwards from Heroku or AppEngine. It's complicated and requires you master its learning curves to deliver benefits. what, exactly, do you think kubernetes is? what is the central design philosohy of it? how does it work? can you name anything about kubernetes that makes it better or different than Heroku or AppEngine? nothing you've said gives me confidenc you can identify the most basic strengths or competencies here, that you can identify how kubernetes is different from the more prosaic & narrow platforms which came before it &nehat that meams. kis is a different take than just an app delivery platform, but i dont see that recognizance here. to me, you are propoing we should all switch from planes to bicycles because bicycles are simpler & all folks need. this reductionism is a misservice to real debate over what & where we should go. endless consumerization of computing, treating all computing workloads as services for petty end-users, who need dumb simple integrated ways to continue shipping legacy software platforms, bundled into fancier containers, is a dead end shithole future. by themselves, simplicity & composability buy us nothing but low-ended-ness. they reduce our ability to think & integrate & fins better cloud patterns to make software with. thankfully i see little potential threat out there from this proposal that we make ourselves more naive. tbis beleif that we can just somehow snap our fingers & simplify & the complexity goes away doesnt seem to ne enough. what i do have faith in are better management tools & practices taht emerge atop & integrating with existing full-blooded clous platforma like kunernetes. folks like ArgoCD or Flux provide a pretty reasonable seveloper experience somewhere reasonably between Herokus bramch-based deployment model & the modern open source cloud platform. last, but definitely not least: what specifically does your comment have to do with this link that was shared? i dont see any direct connection to the link. it seems like you just wanted to dump a generic opinion on us, to neg kubernetes broadly. how if at all have you engaged the material in thia video in your negging? edit: very quickly to -2. im not playing kind but i think silicon deserved heavily to be challenged. if you disagree, i think words are appropriate.

crb 5y ago

If you want to learn the history of Kubernetes from the people that made it, I host a weekly Kubernetes Podcast. If you watched this documentary, you're the target audience. (If you're interested in the docume Many of the people in this documentary have been on the show before; I recommend episodes with Brian Grant, Dawn Chen, Tim Hockin and Clayton Coleman. https://kubernetespodcast.com/ https://kubernetespodcast.com/

pm90 5y ago
+1, great podcast. I used to listen to this one regularly. But the amount of stuff they talked about was just too great to track. So now I listen to specific topics when I may need to use in order to get a informal overview.

deleted 5y ago

[deleted]

hahamaster 5y ago

Oh I was so important, me too, I am very important and we were visionaries solving very important problems that impact millions. It turned out it was very important but it didn't seem that way in retrospect. I'm also very important but also work very well with buzzwords in a way that says absolutely nothing, except that I'm important but we totally didn't know it at the time. None of us wrote a line of code but we were nonetheless very important, and, more importantly, we're even more important now.