4 ms·

The open-source AI boom is built on Big Tech’s handouts. How long will it last?

34 points by AnhTho_FR 3y ago

isawczuk 3y ago
It's symbiotic relationship. OS will survive without Big tech, I'm not sure it works other way around.
- nyokodo 3y ago
  > OS will survive without Big tech, I'm not sure it works other way around. Survive, maybe, but probably not thrive. Big Tech has been a major source of funding of OSS foundations, provider of infrastructure, and employer of many major contributors for decades.
- jchonphoenix 3y ago
  I've been in big tech. They rarely use anything from OSS and usually have their own internal version long before OSS has an offering. Big tech will have no problem thriving without OSS. Their business models and execution aren't reliant on it at all. They only use it for recruiting
  - ke88y 3y ago
    Linux, PyTorch, TensorFlow, literally every programming language used everywhere except perhaps MATLAB and CUDA (and even there most of the libraries are OSS, and those languages aren't FAANG property anyways) I think there's a kernel of truth to what you're saying -- namely that big tech would be fine without oss and mostly builds out their own alternatives whenever necessary or desirable -- but it's definitely not true that BigTech "doesn't use OSS", "has internal versions", or "only use it for recruiting"
    - phyrex 3y ago
      PyTorch is a meta project, I think tensorflow is from Google. Both companies maintain their own programming languages (which they later open sourced)
    - mathisfun123 3y ago
      Lol at thinking that pytorch and tensorflow are oss. I mean they're definitely free but have you ever tried submitting a patch for something that doesn't matter to FB/Google? Those projects function as free bug finding programs for their respective sponsors.
      - vrc 3y ago
        They also exist so that infra/hardware is developed to support them specifically. The more people that use each, the more pressure exists for 3p infra/hw to build around them in pursuit of $$.
      - ke88y 3y ago
        > Lol at thinking that pytorch and tensorflow are oss If BSD isn't open source then I don't know what is. > I mean they're definitely free but have you ever tried submitting a patch for something that doesn't matter to FB/Google? Well, you can always fork. > Those projects function as free bug finding programs for their respective sponsors. Yes, most companies engage with OSS strategically and not purely altruistically. But also: 1. Big tech uses OSS. 2. Big tech does not have internal versions of many OSS projects. 3. Big tech uses OSS for reasons other than just recruiting.
    - jchonphoenix 3y ago
      Meta uses its own php called hack. Nothing is shared with php oss. Google and Meta both have their forked versions of Cpp and python. Plus if they're closed source they will just buy it. Pytorch and tensorflow are from FANG. Without OSS being a thing, they just keep using it without sharing it to the world. Your examples are actually proving the opposite of your point :)
      - ke88y 3y ago
        > Without OSS being a thing, they just keep using it without sharing it to the world. But OSS is a thing and they did share it with the world. > Your examples are actually proving the opposite of your point :) How so? Despite the fact that Big Tech doesn't need OSS, it's not true that BigTech: 1. "doesn't use OSS", or 2. "has internal versions" [of any relevant OSS], or 3. "only use it for recruiting". Hack isn't a counter-example; it's open source. The face that TensorFlow started a google isn't relevant to my point, nor is the fact that FB took the healm of Torch from academia a while ago. Companies engage with OSS strategically, not altruistically. In other news, grass is green :) But there are many strategic reasons for having pieces of infra that are (1) OSS, (2) not duplicated internally, and (3) have motivations other than recruiting.
        deleted 3y ago
        [deleted]
        jchonphoenix 3y ago
        I think you're misunderstanding my comment because you aren't reading it in context of the parent. Those are all points why big tech will be fine without open source software. I.E. if the entirety of the OSS community disappeared tomorrow, big tech would be minimally impacted. Many of the major OSS projects came from then to begin with and they already have massive efforts to maintain parallel versions. They open source tf, hack, and pytorch because it's good for recruiting. Not because it gets commits and reduces workloads. So if the OSS movement disappeared tomorrow, big tech would be the best positioned companies to keep chugging along. Although even if all OSS software that we've discussed here disappeared tomorrow, I know at least Google and Facebook are fine because they don't use any of it. They have their own internal versions that are better and aren't shared with the world.
  - deleted 3y ago
    [deleted]
  - bipop5000 3y ago
    Without OS the talent coming into big tech would probably be severely lesser quality.
  - ralph84 3y ago
    They also use it to manage the pace of innovation in the rest of the industry by open sourcing stuff that is similar but inferior to what they use internally. Hadoop and Kubernetes being prime examples.
- ThrowawayR2 3y ago
  The dotcom boom happened just fine even though open source adoption by businesses was pretty limited at the time. Big tech can afford to pay for software or build their own because they're big; they just like using FOSS because it saves them money.
  - reedciccio 3y ago
    Do you think that Google, the startup of decades ago would have scaled as rapidly if they had to license Solaris and Oracle? Same question for Twitter and Facebook without PHP and Ruby and MySQL and everything else they bootstrapped from?
    - ThrowawayR2 3y ago
      Obviously, yes. I just pointed out the dotcom boom companies as examples of that. Free server hardware would have helped Google, Twitter, Facebook, etc. to scale up faster too but they managed to grow just fine having to pay for servers. Their business models were so incredibly successful that they could have afforded to pay for software too without drastically lowering their growth.
  - zeruch 3y ago
    The dot com boom largely happened because of OS, as it allowed the fast creation/scaling of businesses in a way that would have been impossible otherwise. Big businesses didnt survive the dot com boom and bust because they had bigger budgets. If anything many overextended themselves (e.g. Sun) or missed the boat altogether and were obviated by FOSS (SGI, DEC, et al)
    - ke88y 3y ago
      I'd argue the boom and also the bust happened for that exact reason. And that something similar is happening now with AI. Without OSS we might have seen a more tepid period in the late 90s (smaller boom), as only those who could licenses IBM-whatever or MS-whatever webservers and DBs could make it to market. But also a correspondingly smaller bust, because only those with $$$$ would be able to make the investment, and due to the big barrier to entry likely would've won out their categories. The fact that capabilities were open sourced confused the heck investors who'd been used to the MS/IBM world. They didn't understand how competition would work in the space. Hence terrible investment decisions that in retrospect look hilariously stupid. Don't get me wrong: I much prefer the OSS world. But it's an instructive historical example. Today in AI, any old huckster with a naive nephew who knows how to finetuning things can come to market with something that looks magical to the folks holding the money bags, who likely can't differentiate between trivial hacks on top of open models and a deeper tech moat.
adamnemecek 3y ago
Big Tech is built on open-source. How long will it last?
warning26 3y ago
It's frustrating to me how many people are upset at Meta for releasing Llama (not on HN, but mostly off-HN people.) Like, do you want big tech to control all AI tech? Meta releasing Llama was much better than the alternative.
- ShamelessC 3y ago
  Trying to understand that. How could anyone be upset? Are they AI doomer types?
  - warning26 3y ago
    Yup: https://venturebeat.com/ai/senate-letter-to-meta-on-llama-leak-is-a-threat-to-open-source-ai-at-a-key-moment-say-experts/ https://venturebeat.com/ai/senate-letter-to-meta-on-llama-le...
dcreater 3y ago
Really despise these hackneyed and obviously prejudiced narratives. Open source AI has been the norm all the while. The transformer architecture was by Google but published publicly and this just follows the culture that google, meta and co have been built on. Big tech is not "handing" out anything
otikik 3y ago
Yeah that’s why Google, Meta and Microsoft don’t use any Open Source projects at all and don’t benefit from it at all.
wkat4242 3y ago
I would join a Kickstarter for the training of a real open model. We don't really need handouts.
Havoc 3y ago
Unfortunately it feels a little inevitable that big tech manages to corner this market. H100 cards aren’t exactly enthusiast level gear. On a cold hard cash side the open source model just doesn’t work in that context. Not unless a way is found to distribute training. broadly and efficiently like BOINC / torrent sort of style

34 points by AnhTho_FR 3y ago

isawczuk 3y ago

It's symbiotic relationship. OS will survive without Big tech, I'm not sure it works other way around.

nyokodo 3y ago
> OS will survive without Big tech, I'm not sure it works other way around. Survive, maybe, but probably not thrive. Big Tech has been a major source of funding of OSS foundations, provider of infrastructure, and employer of many major contributors for decades.
jchonphoenix 3y ago
I've been in big tech. They rarely use anything from OSS and usually have their own internal version long before OSS has an offering. Big tech will have no problem thriving without OSS. Their business models and execution aren't reliant on it at all. They only use it for recruiting
- ke88y 3y ago
  Linux, PyTorch, TensorFlow, literally every programming language used everywhere except perhaps MATLAB and CUDA (and even there most of the libraries are OSS, and those languages aren't FAANG property anyways) I think there's a kernel of truth to what you're saying -- namely that big tech would be fine without oss and mostly builds out their own alternatives whenever necessary or desirable -- but it's definitely not true that BigTech "doesn't use OSS", "has internal versions", or "only use it for recruiting"
  - phyrex 3y ago
    PyTorch is a meta project, I think tensorflow is from Google. Both companies maintain their own programming languages (which they later open sourced)
  - mathisfun123 3y ago
    Lol at thinking that pytorch and tensorflow are oss. I mean they're definitely free but have you ever tried submitting a patch for something that doesn't matter to FB/Google? Those projects function as free bug finding programs for their respective sponsors.
    - vrc 3y ago
      They also exist so that infra/hardware is developed to support them specifically. The more people that use each, the more pressure exists for 3p infra/hw to build around them in pursuit of $$.
    - ke88y 3y ago
      > Lol at thinking that pytorch and tensorflow are oss If BSD isn't open source then I don't know what is. > I mean they're definitely free but have you ever tried submitting a patch for something that doesn't matter to FB/Google? Well, you can always fork. > Those projects function as free bug finding programs for their respective sponsors. Yes, most companies engage with OSS strategically and not purely altruistically. But also: 1. Big tech uses OSS. 2. Big tech does not have internal versions of many OSS projects. 3. Big tech uses OSS for reasons other than just recruiting.
  - jchonphoenix 3y ago
    Meta uses its own php called hack. Nothing is shared with php oss. Google and Meta both have their forked versions of Cpp and python. Plus if they're closed source they will just buy it. Pytorch and tensorflow are from FANG. Without OSS being a thing, they just keep using it without sharing it to the world. Your examples are actually proving the opposite of your point :)
    - ke88y 3y ago
      > Without OSS being a thing, they just keep using it without sharing it to the world. But OSS is a thing and they did share it with the world. > Your examples are actually proving the opposite of your point :) How so? Despite the fact that Big Tech doesn't need OSS, it's not true that BigTech: 1. "doesn't use OSS", or 2. "has internal versions" [of any relevant OSS], or 3. "only use it for recruiting". Hack isn't a counter-example; it's open source. The face that TensorFlow started a google isn't relevant to my point, nor is the fact that FB took the healm of Torch from academia a while ago. Companies engage with OSS strategically, not altruistically. In other news, grass is green :) But there are many strategic reasons for having pieces of infra that are (1) OSS, (2) not duplicated internally, and (3) have motivations other than recruiting.
      - deleted 3y ago
        [deleted]
      - jchonphoenix 3y ago
        I think you're misunderstanding my comment because you aren't reading it in context of the parent. Those are all points why big tech will be fine without open source software. I.E. if the entirety of the OSS community disappeared tomorrow, big tech would be minimally impacted. Many of the major OSS projects came from then to begin with and they already have massive efforts to maintain parallel versions. They open source tf, hack, and pytorch because it's good for recruiting. Not because it gets commits and reduces workloads. So if the OSS movement disappeared tomorrow, big tech would be the best positioned companies to keep chugging along. Although even if all OSS software that we've discussed here disappeared tomorrow, I know at least Google and Facebook are fine because they don't use any of it. They have their own internal versions that are better and aren't shared with the world.
- deleted 3y ago
  [deleted]
- bipop5000 3y ago
  Without OS the talent coming into big tech would probably be severely lesser quality.
- ralph84 3y ago
  They also use it to manage the pace of innovation in the rest of the industry by open sourcing stuff that is similar but inferior to what they use internally. Hadoop and Kubernetes being prime examples.
ThrowawayR2 3y ago
The dotcom boom happened just fine even though open source adoption by businesses was pretty limited at the time. Big tech can afford to pay for software or build their own because they're big; they just like using FOSS because it saves them money.
- reedciccio 3y ago
  Do you think that Google, the startup of decades ago would have scaled as rapidly if they had to license Solaris and Oracle? Same question for Twitter and Facebook without PHP and Ruby and MySQL and everything else they bootstrapped from?
  - ThrowawayR2 3y ago
    Obviously, yes. I just pointed out the dotcom boom companies as examples of that. Free server hardware would have helped Google, Twitter, Facebook, etc. to scale up faster too but they managed to grow just fine having to pay for servers. Their business models were so incredibly successful that they could have afforded to pay for software too without drastically lowering their growth.
- zeruch 3y ago
  The dot com boom largely happened because of OS, as it allowed the fast creation/scaling of businesses in a way that would have been impossible otherwise. Big businesses didnt survive the dot com boom and bust because they had bigger budgets. If anything many overextended themselves (e.g. Sun) or missed the boat altogether and were obviated by FOSS (SGI, DEC, et al)
  - ke88y 3y ago
    I'd argue the boom and also the bust happened for that exact reason. And that something similar is happening now with AI. Without OSS we might have seen a more tepid period in the late 90s (smaller boom), as only those who could licenses IBM-whatever or MS-whatever webservers and DBs could make it to market. But also a correspondingly smaller bust, because only those with $$$$ would be able to make the investment, and due to the big barrier to entry likely would've won out their categories. The fact that capabilities were open sourced confused the heck investors who'd been used to the MS/IBM world. They didn't understand how competition would work in the space. Hence terrible investment decisions that in retrospect look hilariously stupid. Don't get me wrong: I much prefer the OSS world. But it's an instructive historical example. Today in AI, any old huckster with a naive nephew who knows how to finetuning things can come to market with something that looks magical to the folks holding the money bags, who likely can't differentiate between trivial hacks on top of open models and a deeper tech moat.

adamnemecek 3y ago

Big Tech is built on open-source. How long will it last?

warning26 3y ago

It's frustrating to me how many people are upset at Meta for releasing Llama (not on HN, but mostly off-HN people.) Like, do you want big tech to control all AI tech? Meta releasing Llama was much better than the alternative.

ShamelessC 3y ago
Trying to understand that. How could anyone be upset? Are they AI doomer types?
- warning26 3y ago
  Yup: https://venturebeat.com/ai/senate-letter-to-meta-on-llama-leak-is-a-threat-to-open-source-ai-at-a-key-moment-say-experts/ https://venturebeat.com/ai/senate-letter-to-meta-on-llama-le...

dcreater 3y ago

Really despise these hackneyed and obviously prejudiced narratives. Open source AI has been the norm all the while. The transformer architecture was by Google but published publicly and this just follows the culture that google, meta and co have been built on. Big tech is not "handing" out anything

otikik 3y ago

Yeah that’s why Google, Meta and Microsoft don’t use any Open Source projects at all and don’t benefit from it at all.

wkat4242 3y ago

I would join a Kickstarter for the training of a real open model. We don't really need handouts.

Havoc 3y ago

Unfortunately it feels a little inevitable that big tech manages to corner this market. H100 cards aren’t exactly enthusiast level gear. On a cold hard cash side the open source model just doesn’t work in that context. Not unless a way is found to distribute training. broadly and efficiently like BOINC / torrent sort of style