8 ms·

Hey, I was just talking about this in a comment the other day! I totally agree with the author here. I was surprised not to see any mention of cloud facing open

Hey, I was just talking about this in a comment the other day! I totally agree with the author here. I was surprised not to see any mention of cloud facing open source products. Kubernetes seems like a prime example. Google lost its containerization first mover advantage to Docker (remember google was famous early for its internal use of containers). So they targeted the next level up the stack of container orchestration. Kubernetes is a great project, but its clear priority is compatibility with the cloud computing business model. It didn’t take long for google to offer GKE on top of kubernetes. At least for now, there is feature parity between open source and hosted versions. But I don’t expect that to last long. I would be wary of the kubernetes project introducing features that only work well in google’s cloud. The main externality of this open source EEE strategy is developer opportunity cost. When google swept into the chat market, they used their size and first mover advantage to push an open standard, XMPP. So developers who might have worked on other projects/standards naturally gravitated toward XMPP and contributed their time there. If those developers had known the future of XMPP, they likely would not have worked on it. They could have invested time in a competing, more open project instead. This to me is the real danger. It’s bad enough that google gets to benefit from thousands of free contributors to its “open source” code. It’s even worse that they are taking those developers away from other projects, so that when google moves to the third phase of EEE, there are no viable alternatives.

jacksmith21006 9y ago
Google completely gave away K8s including even the brand. So not sure what you think Google will do? They are giving away everything they learned with Borg and you somehow take it as a negative? Really? Do think from a business standpoint not sure why Google would give up the K8s brand but it is good for everyone. I really worry that Google will stop opens sourcing and more important sharing so much through papers when people b*tch so much. Starting to be hard to see why they even do it any longer. Who else have a huge competitive advantage with something like Borg and just gives it away to the industry? But I am someone super curious and just probably spend more time on https://research.google.com/pubs/papers.html https://research.google.com/pubs/papers.html Read most of the paper. But then what they presented at NIPS. My favorite paper right now is https://arxiv.org/abs/1712.01208 https://arxiv.org/abs/1712.01208 Bigger example is how much they give away in terms of AI/ML.
- hellepardo 9y ago
  > Bigger example is how much they give away in terms of AI/ML. You think they are publishing all these papers and open-sourcing TensorFlow for the good of the community solely? No... if the entire ML community publishes papers with TensorFlow as code, then now Google can sell that many more TPUs and that much more access to their cloud systems to run the models...
  - jacksmith21006 9y ago
    No not solely. But often it is mutual benefit. But the one that seems most crazy is giving away Borg. I was shocked they gave away the K8s brand. Now that was ballsy and surprised necessary.
- LamaOfRuin 9y ago
  >Do think from a business standpoint not sure why Google would give up the K8s brand but it is good for everyone. Because with Borg they only had an advantage with their own products' operations. That's nice, in that they are able to maintain a decent amount of technical superiority and have their product reliability be leading or tied for first, but it's not a clear line between this and them generating more revenue. They want their cloud offering to compete with AWS and Azure, which is why they want K8s to gain mindshare. GCE has huge potential for growth and significant switching costs; meaning they need to get new products on it sooner rather than later, and developer mindshare is the way to do that.
  - jacksmith21006 9y ago
    I listen to a interview with a higher level Googler that explained why and it made a ton of sense to me. They were tired of writing something up as a paper and then someone implemented as OSS and then companies and engineers knew the OSS implementation instead of their own. So instead do a OSS version and then drive what everyone learns. I found this a bit funny. One solution would have been to not write the papers. But so glad Google does. Spend a lot of time on their research site. https://research.google.com/pubs/papers.html https://research.google.com/pubs/papers.html
hellepardo 9y ago
I have always felt that, in a "typical" open-source project, I could approach the community, join it, build some rapport, and eventually help with the decision-making process (assuming I was any good and people in the community liked me). But with Google-run (or other company-run) open-source projects, it's a completely different story. Want to be a TensorFlow maintainer? Great, sure, you can contribute, but you'll never get to make any decisions about the direction of the project itself unless you're inside Google. To me it's a disappointing semi-abuse of the term "open source"... it doesn't mean what I think it should...
- FLUX-YOU 9y ago
  If you are good enough to contribute to Tensorflow and be able to come up with new directions, why not just get a job at Google so you can do that? This isn't unlike any other open source project ran by people who don't want to consider all input from others. Not all suggestions are worth serious consideration, after all. Open source doesn't mean I'm beholden to your every whim just because you've contributed. That's what forks are for.
  - hellepardo 9y ago
    > why not just get a job at Google so you can do that? Don't want to move to California? Can't pass the interview? Salary too low in the offer? There could be lots of reasons. I agree with your other points, but what I'm saying is not exactly that. What I'm saying is that TensorFlow is not a community-led project in the way that other open-source projects are. It's Google-led, and if you don't like it, you can go away or fork it (...and if you fork it, it's unlikely you'll get many users given the Google marketing machine). (The same complaints have been and are being regularly made against Red Hat, and also Fedora, given that Fedora is basically "experimental" Red Hat.)
    - FLUX-YOU 9y ago
      Convince Google it's better as a community-led project. Maybe they will have some convincing counter-points about why it's better the way that it is that we're not considering. They've likely already been through this debate. Consider if Tensorflow were half as useful as it is today if it was community-led and it was led by a shitty community.
kogir 9y ago
I don’t understand your XMPP example. Google didn’t and doesn’t decide the fate of XMPP. Most developers who targeted it only cared about where the users were, and features supported by the most popular servers and clients. Would you have preferred that those developers reverse engineer something closed source Google developed instead? Libpurple has advanced messaging not at all, only been great for interoperability.
- dontchooseanick 9y ago
  I've worked on XMPP. Google talk / now hangouts once was an XMPP pusher. XMPP is decentralized. As with mail servers you can advertise your own XMPP servers, see https://wiki.xmpp.org/web/SRV_Records https://wiki.xmpp.org/web/SRV_Records , you can even publish a per-user DNS record with an OTR key. I shut down my XMPP endpoint a few monthes ago because Google had stopped supporting XMPP bridging to hangouts. Now that everybody uses hangouts (coders) or slack (teams/corporates) or whatsapp/messenger/skype (everyone else) there is no incentive to support it anymore (I speak now only IRC with coders) They gave their support and a lot of traction to the protocol, and removed it on a second step (once advocates of XMPP, _yes, like me_ said to everyone, "yes you should prefer gmail+talk, they are supporting open standards") Looks like an approaching-with-good-intentions and then disrupting strategy to me. Even if you disagree on the responsibility of XMPP rise and fall, they are the bull in a china shop : don't drink the kool aid, you'll end up broken.
- jononor 9y ago
  Gmail/Google was one of the biggest platforms that supported XMPP. Then they removed it completely, with of course made supporting XMPP less meaningful/attractive for others.
jchw 9y ago
Call me naive but I disagree with the points about Kubernetes. While they did make Kubernetes specifically for their GKE offering, it was before Docker Swarm even existed, so it didn't directly compete with Docker. I doubt Swarm was doomed to failure; more, it felt like the priorities were off (it was simpler than Kubernetes at the cost of being less useful.) Further, Google has been taking the Google bits out of Kubernetes. They declined to call their own offering "Google Kubernetes Engine" officially until the playing field was level and they had a certification program. Amazon, Azure, DigitalOcean and others are implementing their own managed Kubernetes solutions. Google doesn't have to cheat here. Speaking as someone dealing with Google Cloud directly, they are mainly hitting on the point that Google Cloud offers better network and disk performance, in a more flexible manner, for significantly less money. And honest to God they're right. The CPU performance is worse in some cases, but we're saving money on workloads so far. On the flip side, as long time AWS users, the Kubernetes experience has been pretty good since Kubernetes 1.3. New features typically support AWS pretty early in, with the main exception being Ingress controllers (for good reason: ALBs weren't even released when that became a thing.) Google is one of those companies big enough to make it easy to feel like they have evil ulterior motives, but I'll be honest: in my dealings with them, I've felt that the opposite is true and generally the intentions are good.
- scarface74 9y ago
  I wouldn't trust my business to Google Cloud Platform because they have a habit of abandoning services that aren't related to their core business - search and ads. I have more confidence in Amazon and Microsoft.
  - jchw 9y ago
    I'm sorry you feel that way, because Google Cloud is really nice and if nothing else, it provokes a lot of thought. It's worth evaluating in my opinion. I'm not saying it's better than other offerings - It's not all around - but the building blocks are really nice.
    - scarface74 9y ago
bryanlarsen 9y ago
IMO Kubernetes is safe. It has way too many outside contributors for Google to pull it back in. And it's going in the opposite direction as you describe: networking, storage and ingress used to be considerably more clunky on non-GKE based installations, but now we're at the stage where people are starting to use third-party ingress and storage plugins on GKE because those have features that the Google implementation doesn't have. And Google is probably ecstatic. It's disrupting the AWS monopoly that was developing, tons of outside developers are improving GKE for them, and it's running a textbook "commoditize your complements" play. Compare that with Android, which Google is successfully doing the EEE strategy on. Despite widespread usage, it has few outside contributors, so Google is bearing all of the costs of it being open source without reaping few of the benefits. But that's Google's fault. It's hard to build and contribute to Android. Development is done privately and thrown over the wall. OTOH, it's easy to contribute to Kubernetes, and it's done in the open.
- scrollaway 9y ago
  Posted further down, but I'll move it here since it's more relevant to android's development: So a few years ago I created an open source community in gaming (which coincidentally, the author of this blog post was a part of early on). We had a Github organization, an IRC channel, etc. It was fun. At some point I grew part of the community into a company, which today is growing and thriving. We're rooted in open source. I'm rooted in open source (Nearly all my work has been public and on Github for the past decade). Today we still have nearly all our work up and public on Github but get next to zero external contributions. Because contributing as someone external to the company is hard. Unless your actual goal as the company is to produce free & open source software, it's going to be really, really hard to make it possible for external actors to contribute to the core products as smoothly as for the company's employees. There's loads of things that pile up, like reusing internal CI systems for testing, requiring API keys and passwords for access to resources that are required or highly useful for development (eg. testing a redshift environment), and so on. All those things are fixable, but take a huge amount of time to fix, and why would you fix it if nearly nobody external to the company is contributing anyway? It's not like making the experience easy is a guaranteed win either. Things like product design, project management, etc... it's not obvious how to make those open processes. It's not like you can do part time project management contributions to any random open source project out there. And when the processes aren't open, the barrier of entry to contribution is that much higher. And quite frankly, certain people (who will recognize themselves) make it extremely unappealing to spend a lot of efforts on keeping the entire project open. The kind of people who will complain about every little detail of how you're trying to open up the project, until its name is prefixed with GNU/. "All or nothing" is an atrocious approach to open source and actively harms the ecosystem. Point is, don't blame Google for making Android and Chromium hard to contribute to. It's not like it's on purpose, it just takes a huge investment for it to be possible. I mean fuck, Firefox isn't exactly easy to contribute to, and Mozilla is one of the most open companies out there, and in many ways a role model of how an open source company should be run.
seba_dos1 9y ago
Uhm... Google wasn't the first to use XMPP. XMPP is also as open as a project can be. It's still going strong, providing a modern, mobile friendly IM protocol - it just happens without Google now. XMPP never was a Google project of any kind. They just played EEE by having the biggest server and making it incompatible with the rest of the network. That's a completely different story that what it seems from your post - no external development happened to progress Google's service in any way. Google backstabbed XMPP, sure, but it's healing pretty well. I don't see how any time spend by developers on XMPP is wasted just because Google backed off. They took their userbase with them, sure, but if you want modern, open, federated IM network, XMPP is a sane choice. The only proper alternative these days is Matrix, but that's more of an IRC replacement.

jacksmith21006 9y ago

Google completely gave away K8s including even the brand. So not sure what you think Google will do? They are giving away everything they learned with Borg and you somehow take it as a negative? Really? Do think from a business standpoint not sure why Google would give up the K8s brand but it is good for everyone. I really worry that Google will stop opens sourcing and more important sharing so much through papers when people b*tch so much. Starting to be hard to see why they even do it any longer. Who else have a huge competitive advantage with something like Borg and just gives it away to the industry? But I am someone super curious and just probably spend more time on https://research.google.com/pubs/papers.html https://research.google.com/pubs/papers.html Read most of the paper. But then what they presented at NIPS. My favorite paper right now is https://arxiv.org/abs/1712.01208 https://arxiv.org/abs/1712.01208 Bigger example is how much they give away in terms of AI/ML.

hellepardo 9y ago
> Bigger example is how much they give away in terms of AI/ML. You think they are publishing all these papers and open-sourcing TensorFlow for the good of the community solely? No... if the entire ML community publishes papers with TensorFlow as code, then now Google can sell that many more TPUs and that much more access to their cloud systems to run the models...
- jacksmith21006 9y ago
  No not solely. But often it is mutual benefit. But the one that seems most crazy is giving away Borg. I was shocked they gave away the K8s brand. Now that was ballsy and surprised necessary.
LamaOfRuin 9y ago
>Do think from a business standpoint not sure why Google would give up the K8s brand but it is good for everyone. Because with Borg they only had an advantage with their own products' operations. That's nice, in that they are able to maintain a decent amount of technical superiority and have their product reliability be leading or tied for first, but it's not a clear line between this and them generating more revenue. They want their cloud offering to compete with AWS and Azure, which is why they want K8s to gain mindshare. GCE has huge potential for growth and significant switching costs; meaning they need to get new products on it sooner rather than later, and developer mindshare is the way to do that.
- jacksmith21006 9y ago
  I listen to a interview with a higher level Googler that explained why and it made a ton of sense to me. They were tired of writing something up as a paper and then someone implemented as OSS and then companies and engineers knew the OSS implementation instead of their own. So instead do a OSS version and then drive what everyone learns. I found this a bit funny. One solution would have been to not write the papers. But so glad Google does. Spend a lot of time on their research site. https://research.google.com/pubs/papers.html https://research.google.com/pubs/papers.html

hellepardo 9y ago

I have always felt that, in a "typical" open-source project, I could approach the community, join it, build some rapport, and eventually help with the decision-making process (assuming I was any good and people in the community liked me). But with Google-run (or other company-run) open-source projects, it's a completely different story. Want to be a TensorFlow maintainer? Great, sure, you can contribute, but you'll never get to make any decisions about the direction of the project itself unless you're inside Google. To me it's a disappointing semi-abuse of the term "open source"... it doesn't mean what I think it should...

FLUX-YOU 9y ago
If you are good enough to contribute to Tensorflow and be able to come up with new directions, why not just get a job at Google so you can do that? This isn't unlike any other open source project ran by people who don't want to consider all input from others. Not all suggestions are worth serious consideration, after all. Open source doesn't mean I'm beholden to your every whim just because you've contributed. That's what forks are for.
- hellepardo 9y ago
  > why not just get a job at Google so you can do that? Don't want to move to California? Can't pass the interview? Salary too low in the offer? There could be lots of reasons. I agree with your other points, but what I'm saying is not exactly that. What I'm saying is that TensorFlow is not a community-led project in the way that other open-source projects are. It's Google-led, and if you don't like it, you can go away or fork it (...and if you fork it, it's unlikely you'll get many users given the Google marketing machine). (The same complaints have been and are being regularly made against Red Hat, and also Fedora, given that Fedora is basically "experimental" Red Hat.)
  - FLUX-YOU 9y ago
    Convince Google it's better as a community-led project. Maybe they will have some convincing counter-points about why it's better the way that it is that we're not considering. They've likely already been through this debate. Consider if Tensorflow were half as useful as it is today if it was community-led and it was led by a shitty community.

kogir 9y ago

I don’t understand your XMPP example. Google didn’t and doesn’t decide the fate of XMPP. Most developers who targeted it only cared about where the users were, and features supported by the most popular servers and clients. Would you have preferred that those developers reverse engineer something closed source Google developed instead? Libpurple has advanced messaging not at all, only been great for interoperability.

dontchooseanick 9y ago
I've worked on XMPP. Google talk / now hangouts once was an XMPP pusher. XMPP is decentralized. As with mail servers you can advertise your own XMPP servers, see https://wiki.xmpp.org/web/SRV_Records https://wiki.xmpp.org/web/SRV_Records , you can even publish a per-user DNS record with an OTR key. I shut down my XMPP endpoint a few monthes ago because Google had stopped supporting XMPP bridging to hangouts. Now that everybody uses hangouts (coders) or slack (teams/corporates) or whatsapp/messenger/skype (everyone else) there is no incentive to support it anymore (I speak now only IRC with coders) They gave their support and a lot of traction to the protocol, and removed it on a second step (once advocates of XMPP, _yes, like me_ said to everyone, "yes you should prefer gmail+talk, they are supporting open standards") Looks like an approaching-with-good-intentions and then disrupting strategy to me. Even if you disagree on the responsibility of XMPP rise and fall, they are the bull in a china shop : don't drink the kool aid, you'll end up broken.
jononor 9y ago
Gmail/Google was one of the biggest platforms that supported XMPP. Then they removed it completely, with of course made supporting XMPP less meaningful/attractive for others.

jchw 9y ago

Call me naive but I disagree with the points about Kubernetes. While they did make Kubernetes specifically for their GKE offering, it was before Docker Swarm even existed, so it didn't directly compete with Docker. I doubt Swarm was doomed to failure; more, it felt like the priorities were off (it was simpler than Kubernetes at the cost of being less useful.) Further, Google has been taking the Google bits out of Kubernetes. They declined to call their own offering "Google Kubernetes Engine" officially until the playing field was level and they had a certification program. Amazon, Azure, DigitalOcean and others are implementing their own managed Kubernetes solutions. Google doesn't have to cheat here. Speaking as someone dealing with Google Cloud directly, they are mainly hitting on the point that Google Cloud offers better network and disk performance, in a more flexible manner, for significantly less money. And honest to God they're right. The CPU performance is worse in some cases, but we're saving money on workloads so far. On the flip side, as long time AWS users, the Kubernetes experience has been pretty good since Kubernetes 1.3. New features typically support AWS pretty early in, with the main exception being Ingress controllers (for good reason: ALBs weren't even released when that became a thing.) Google is one of those companies big enough to make it easy to feel like they have evil ulterior motives, but I'll be honest: in my dealings with them, I've felt that the opposite is true and generally the intentions are good.

scarface74 9y ago
I wouldn't trust my business to Google Cloud Platform because they have a habit of abandoning services that aren't related to their core business - search and ads. I have more confidence in Amazon and Microsoft.
- jchw 9y ago
  I'm sorry you feel that way, because Google Cloud is really nice and if nothing else, it provokes a lot of thought. It's worth evaluating in my opinion. I'm not saying it's better than other offerings - It's not all around - but the building blocks are really nice.
  - scarface74 9y ago

bryanlarsen 9y ago

IMO Kubernetes is safe. It has way too many outside contributors for Google to pull it back in. And it's going in the opposite direction as you describe: networking, storage and ingress used to be considerably more clunky on non-GKE based installations, but now we're at the stage where people are starting to use third-party ingress and storage plugins on GKE because those have features that the Google implementation doesn't have. And Google is probably ecstatic. It's disrupting the AWS monopoly that was developing, tons of outside developers are improving GKE for them, and it's running a textbook "commoditize your complements" play. Compare that with Android, which Google is successfully doing the EEE strategy on. Despite widespread usage, it has few outside contributors, so Google is bearing all of the costs of it being open source without reaping few of the benefits. But that's Google's fault. It's hard to build and contribute to Android. Development is done privately and thrown over the wall. OTOH, it's easy to contribute to Kubernetes, and it's done in the open.

scrollaway 9y ago
Posted further down, but I'll move it here since it's more relevant to android's development: So a few years ago I created an open source community in gaming (which coincidentally, the author of this blog post was a part of early on). We had a Github organization, an IRC channel, etc. It was fun. At some point I grew part of the community into a company, which today is growing and thriving. We're rooted in open source. I'm rooted in open source (Nearly all my work has been public and on Github for the past decade). Today we still have nearly all our work up and public on Github but get next to zero external contributions. Because contributing as someone external to the company is hard. Unless your actual goal as the company is to produce free & open source software, it's going to be really, really hard to make it possible for external actors to contribute to the core products as smoothly as for the company's employees. There's loads of things that pile up, like reusing internal CI systems for testing, requiring API keys and passwords for access to resources that are required or highly useful for development (eg. testing a redshift environment), and so on. All those things are fixable, but take a huge amount of time to fix, and why would you fix it if nearly nobody external to the company is contributing anyway? It's not like making the experience easy is a guaranteed win either. Things like product design, project management, etc... it's not obvious how to make those open processes. It's not like you can do part time project management contributions to any random open source project out there. And when the processes aren't open, the barrier of entry to contribution is that much higher. And quite frankly, certain people (who will recognize themselves) make it extremely unappealing to spend a lot of efforts on keeping the entire project open. The kind of people who will complain about every little detail of how you're trying to open up the project, until its name is prefixed with GNU/. "All or nothing" is an atrocious approach to open source and actively harms the ecosystem. Point is, don't blame Google for making Android and Chromium hard to contribute to. It's not like it's on purpose, it just takes a huge investment for it to be possible. I mean fuck, Firefox isn't exactly easy to contribute to, and Mozilla is one of the most open companies out there, and in many ways a role model of how an open source company should be run.

seba_dos1 9y ago

Uhm... Google wasn't the first to use XMPP. XMPP is also as open as a project can be. It's still going strong, providing a modern, mobile friendly IM protocol - it just happens without Google now. XMPP never was a Google project of any kind. They just played EEE by having the biggest server and making it incompatible with the rest of the network. That's a completely different story that what it seems from your post - no external development happened to progress Google's service in any way. Google backstabbed XMPP, sure, but it's healing pretty well. I don't see how any time spend by developers on XMPP is wasted just because Google backed off. They took their userbase with them, sure, but if you want modern, open, federated IM network, XMPP is a sane choice. The only proper alternative these days is Matrix, but that's more of an IRC replacement.