5 ms·

Having enough scale to perpetually offer free/low-cost compute is a moat. The primary reason ChatGPT went viral in the first place was because it was free, with

Having enough scale to perpetually offer free/low-cost compute is a moat. The primary reason ChatGPT went viral in the first place was because it was free, with no restrictions. Back in 2019, GPT-2 1.5B was made freely accessible by a single developer via the TalkToTransformers website, which was the very first time many people talking about AI text generation...then the owner got hit with sticker shock from the GPU compute needed to scale. AI text generation competitors like Cohere and Anthropic will never be able to compete with Microsoft/Google/Amazon on marginal cost.

FemmeAndroid 4y ago
Charity is only a moat if it’s not profitable.
- r00fus 4y ago
  There's "immediately profitable" and "eventually profitable". Vast compute scale allows collection of customer generated data so the latter is possible, AI as of yet is not the former. So GP point still stands. FAAMG can run much larger immediate deficits in order to corner the market on the eventual profitability of AI.
  - Tostino 4y ago
    The amount of valuable data generated from professionals using these services to work through their problems and find solutions to industry problems is immense. It essentially gives these companies the keys to automating many industries by just...letting people try and make their jobs easier and collecting all data.
  - cushpush 4y ago
    All this talk that every investment pays off in the end is faulty and dangerous. Many investments don't pan out, 95% of the firms you see in the ticker this decade might be gone, and yet everyone is very confident is underwriting these "losses for future gains" but really it's economies of scale. It doesn't cost MSFT much more to run the GPU than to turn it on in the first place.
  - indymike 4y ago
    > . FAAMG can run much larger immediate deficits in order to corner the market on the eventual profitability of AI. This assumes that there is a corner-able market. Previously, the cost of training was the moat. That appears to have been more of a puddle under the gate than an actual moat.
- deelowe 4y ago
  It seems the plan is to be a loss leader until scale is sufficient to reach near AGI levels of capability.
  - nirav72 4y ago
    There was some indication recently that OpenAI was spending over $500k/day to keep it running. Not sure how long thats going to last. AGI is still a pipe dream. Sooner or later , they’re going to have to make money.
    - cmelbye 4y ago
      Assuming you're talking about the free ChatGPT product, it's important to consider the value of the training data that users are giving them. Beyond that, they are making a lot of money from their enterprise offerings (API products, custom partnerships, etc.) with more to come soon, like ChatGPT for Business.
      - phatfish 4y ago
        I know there are use cases out there, so it's not a dig. I'm curious how many enterprises are actually spending money with OpenAI right now to do internal development. Have they released any figures?
    - blihp 4y ago
      Oh no, they're going belly up in 20,000 days! (i.e. $10B / 500k) Compute is going to keep getting cheaper and they're going to keep optimizing it to reduce how much compute it needs. I'm more curious about their next steps rather than how they're going to keep the lights on for ChatGPT.
      - adrianmonk 4y ago
        https://www.youtube.com/watch?v=z9OUZNicTGU&t=123s https://www.youtube.com/watch?v=z9OUZNicTGU&t=123s
    - 4y ago
- moron4hire 4y ago
  In other words, engage in anti-competitive behavior.
- sharemywin 4y ago
  This is the timeline that's scaring the shit out of them: Feb 24, 2023: Meta launches LLaMA, a relatively small, open-source AI model. March 3, 2023: LLaMA is leaked to the public, spurring rapid innovation. March 12, 2023: Artem Andreenko runs LLaMA on a Raspberry Pi, inspiring minification efforts. March 13, 2023: Stanford's Alpaca adds instruction tuning to LLaMA, enabling low-budget fine-tuning. March 18, 2023: Georgi Gerganov's 4-bit quantization enables LLaMA to run on a MacBook CPU. March 19, 2023: Vicuna, a 13B model, achieves "parity" with Bard at a $300 training cost. March 25, 2023: Nomic introduces GPT4All, an ecosystem gathering models like Vicuna at a $100 training cost. March 28, 2023: Cerebras trains an open-source GPT-3 architecture, making the community independent of LLaMA. March 28, 2023: LLaMA-Adapter achieves SOTA multimodal ScienceQA with 1.2M learnable parameters. April 3, 2023: Berkeley's Koala dialogue model rivals ChatGPT in user preference at a $100 training cost. April 15, 2023: Open Assistant releases an open-source RLHF model and dataset, making alignment more accessible.
  - sharemywin 4y ago
    this doesn't even include the stuff around agents and/or langchain
    - politician 4y ago
      The post mentions that they consider "Responsible Release" to be an unsolved hard problem internally. It's possible that they are culturally blind to agents.
      - PoignardAzur 4y ago
        They're basically saying that Pandora's Box, assuming it exists, has already been open. Even if OpenAI, Facebook AI Research and Google DeepMind all shut down tomorrow, research capable of producing agents will continue worldwide.
  - ByThyGrace 4y ago
    Interesting! It's like nothing has happened on the field for the last three weeks heh
dragonwriter 4y ago
> Having enough scale to perpetually offer free/low-cost compute is a moat. Its a moat for services, not models, and its only a moat for AI services as long as that compute isn’t hobbled by being used for models which are so inefficient compared to SOTA as to waste the advantage, which underlines why leaning into open source the way this piece urges is in Google’s interests, the same way open source has worked to Google and Amazon’s benefits as service providers in other domains. (Not so much “the ability to offer free/low-cost compute” but “the advantages of scale and existing need for widely geographically dispersed compute on the cost of both marginal compute and having marginal compute close to the customer where that is relevant”, but those are pretty close to differenly-focussed rephrasings of the same underlying reality.)
seydor 4y ago
That's what a lot of people think until they run Vicuna 13B or equivalent. We're just 5 months in this, there will be many leaps.
- bilbo0s 4y ago
  Yes there will, that's the problem HN User Minimaxir is talking about. It will only get less and less expensive for Microsoft in terms of cost. And more and more effective for Microsoft in terms of results delivered. How do you compete with free? That's the question. The previous internet experience has already shown us that "also be free" is not really a sustainable or even effective answer. You have to be better in some fundamental dimension.
  - deleted 4y ago
    [deleted]
  - deleted 4y ago
    [deleted]
- BiteCode_dev 4y ago
  What makes you think open ai won't look at the FOSS improvements, include them in their tech, and make their GPU farm way cheaper, rendering their service even more competitive? Not to mention it's easy to run stable diffusion, but midjourney is still a good business. I can run sd on my laptop, I still pay for midjourney because it's convenient, the out of the box experience is better than any competition, and it keeps improving.
  - seydor 4y ago
    that's like saying that apple and MS can look into linux and steal ideas. Yes they can do that but it doesnt make linux any less useful. If anything they learned to contribute back to the common pile, because everyone benefits from it. It would be a problem if this was a one-way relationship , which it doesnt seem to be. If Open source is making them more money, why kill it
BiteCode_dev 4y ago
And ChatGPT has a super low barrier to entry while open source alternatives have a high one. Creating a service that can compete with it on that regard implies you can scale GPU farms in a cost effective way. It's not as easy as it sounds. Meanwhile, openai still improves their product very fast, and unlike google, it's their only one. It's their baby. It has their entire focus. Since for most consumers, AI == ChatGPT, they have the best market share right now, which mean the most user feedback to improve their product. Which they do at a fast pace. They also understand that to get mass adoption, they need to censor the AI, like MacDonald and Disney craft their family friendly image. Which irritate every geeks, including me, but make commercially sense. Plus, despite the fact you can torrent music and watch it with VLC, and that Amazon+Disney are competitors, netflix exists. Having a quality service has value in itself. I would not count open ai as dead as a lot of people seem to desperately want it to be. Just because Google missed the AI train doesn't mean wishful thinking the market to be killed by FOSS is going to make it so. As usual with those things it's impossible to know in advance what's going to happen, but odds are not disfavoring chatgpt as much as this article says.
bickfordb 4y ago
A good example of this is Youtube
freediver 4y ago
> AI text generation competitors like Cohere and Anthropic will never be able to compete with Microsoft/Google/Amazon on marginal cost. Anthropic already does, with its models. They are same price or cheaper than OpenAI, with comparable quality. > Having enough scale to perpetually offer free/low-cost compute is a moat. Rather than a moat it is a growth strategy. At one point in time you need to start to monetize and this is the moment when rubber hits the road. If you can survive monetization and continue to grow, now you have a moat.

FemmeAndroid 4y ago

Charity is only a moat if it’s not profitable.

r00fus 4y ago
There's "immediately profitable" and "eventually profitable". Vast compute scale allows collection of customer generated data so the latter is possible, AI as of yet is not the former. So GP point still stands. FAAMG can run much larger immediate deficits in order to corner the market on the eventual profitability of AI.
- Tostino 4y ago
  The amount of valuable data generated from professionals using these services to work through their problems and find solutions to industry problems is immense. It essentially gives these companies the keys to automating many industries by just...letting people try and make their jobs easier and collecting all data.
- cushpush 4y ago
  All this talk that every investment pays off in the end is faulty and dangerous. Many investments don't pan out, 95% of the firms you see in the ticker this decade might be gone, and yet everyone is very confident is underwriting these "losses for future gains" but really it's economies of scale. It doesn't cost MSFT much more to run the GPU than to turn it on in the first place.
- indymike 4y ago
  > . FAAMG can run much larger immediate deficits in order to corner the market on the eventual profitability of AI. This assumes that there is a corner-able market. Previously, the cost of training was the moat. That appears to have been more of a puddle under the gate than an actual moat.
deelowe 4y ago
It seems the plan is to be a loss leader until scale is sufficient to reach near AGI levels of capability.
- nirav72 4y ago
  There was some indication recently that OpenAI was spending over $500k/day to keep it running. Not sure how long thats going to last. AGI is still a pipe dream. Sooner or later , they’re going to have to make money.
  - cmelbye 4y ago
    Assuming you're talking about the free ChatGPT product, it's important to consider the value of the training data that users are giving them. Beyond that, they are making a lot of money from their enterprise offerings (API products, custom partnerships, etc.) with more to come soon, like ChatGPT for Business.
    - phatfish 4y ago
      I know there are use cases out there, so it's not a dig. I'm curious how many enterprises are actually spending money with OpenAI right now to do internal development. Have they released any figures?
  - blihp 4y ago
    Oh no, they're going belly up in 20,000 days! (i.e. $10B / 500k) Compute is going to keep getting cheaper and they're going to keep optimizing it to reduce how much compute it needs. I'm more curious about their next steps rather than how they're going to keep the lights on for ChatGPT.
    - adrianmonk 4y ago
      https://www.youtube.com/watch?v=z9OUZNicTGU&t=123s https://www.youtube.com/watch?v=z9OUZNicTGU&t=123s
  - 4y ago
moron4hire 4y ago
In other words, engage in anti-competitive behavior.
sharemywin 4y ago
This is the timeline that's scaring the shit out of them: Feb 24, 2023: Meta launches LLaMA, a relatively small, open-source AI model. March 3, 2023: LLaMA is leaked to the public, spurring rapid innovation. March 12, 2023: Artem Andreenko runs LLaMA on a Raspberry Pi, inspiring minification efforts. March 13, 2023: Stanford's Alpaca adds instruction tuning to LLaMA, enabling low-budget fine-tuning. March 18, 2023: Georgi Gerganov's 4-bit quantization enables LLaMA to run on a MacBook CPU. March 19, 2023: Vicuna, a 13B model, achieves "parity" with Bard at a $300 training cost. March 25, 2023: Nomic introduces GPT4All, an ecosystem gathering models like Vicuna at a $100 training cost. March 28, 2023: Cerebras trains an open-source GPT-3 architecture, making the community independent of LLaMA. March 28, 2023: LLaMA-Adapter achieves SOTA multimodal ScienceQA with 1.2M learnable parameters. April 3, 2023: Berkeley's Koala dialogue model rivals ChatGPT in user preference at a $100 training cost. April 15, 2023: Open Assistant releases an open-source RLHF model and dataset, making alignment more accessible.
- sharemywin 4y ago
  this doesn't even include the stuff around agents and/or langchain
  - politician 4y ago
    The post mentions that they consider "Responsible Release" to be an unsolved hard problem internally. It's possible that they are culturally blind to agents.
    - PoignardAzur 4y ago
      They're basically saying that Pandora's Box, assuming it exists, has already been open. Even if OpenAI, Facebook AI Research and Google DeepMind all shut down tomorrow, research capable of producing agents will continue worldwide.
- ByThyGrace 4y ago
  Interesting! It's like nothing has happened on the field for the last three weeks heh

dragonwriter 4y ago

> Having enough scale to perpetually offer free/low-cost compute is a moat. Its a moat for services, not models, and its only a moat for AI services as long as that compute isn’t hobbled by being used for models which are so inefficient compared to SOTA as to waste the advantage, which underlines why leaning into open source the way this piece urges is in Google’s interests, the same way open source has worked to Google and Amazon’s benefits as service providers in other domains. (Not so much “the ability to offer free/low-cost compute” but “the advantages of scale and existing need for widely geographically dispersed compute on the cost of both marginal compute and having marginal compute close to the customer where that is relevant”, but those are pretty close to differenly-focussed rephrasings of the same underlying reality.)

seydor 4y ago

That's what a lot of people think until they run Vicuna 13B or equivalent. We're just 5 months in this, there will be many leaps.

bilbo0s 4y ago
Yes there will, that's the problem HN User Minimaxir is talking about. It will only get less and less expensive for Microsoft in terms of cost. And more and more effective for Microsoft in terms of results delivered. How do you compete with free? That's the question. The previous internet experience has already shown us that "also be free" is not really a sustainable or even effective answer. You have to be better in some fundamental dimension.
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]
BiteCode_dev 4y ago
What makes you think open ai won't look at the FOSS improvements, include them in their tech, and make their GPU farm way cheaper, rendering their service even more competitive? Not to mention it's easy to run stable diffusion, but midjourney is still a good business. I can run sd on my laptop, I still pay for midjourney because it's convenient, the out of the box experience is better than any competition, and it keeps improving.
- seydor 4y ago
  that's like saying that apple and MS can look into linux and steal ideas. Yes they can do that but it doesnt make linux any less useful. If anything they learned to contribute back to the common pile, because everyone benefits from it. It would be a problem if this was a one-way relationship , which it doesnt seem to be. If Open source is making them more money, why kill it

BiteCode_dev 4y ago

And ChatGPT has a super low barrier to entry while open source alternatives have a high one. Creating a service that can compete with it on that regard implies you can scale GPU farms in a cost effective way. It's not as easy as it sounds. Meanwhile, openai still improves their product very fast, and unlike google, it's their only one. It's their baby. It has their entire focus. Since for most consumers, AI == ChatGPT, they have the best market share right now, which mean the most user feedback to improve their product. Which they do at a fast pace. They also understand that to get mass adoption, they need to censor the AI, like MacDonald and Disney craft their family friendly image. Which irritate every geeks, including me, but make commercially sense. Plus, despite the fact you can torrent music and watch it with VLC, and that Amazon+Disney are competitors, netflix exists. Having a quality service has value in itself. I would not count open ai as dead as a lot of people seem to desperately want it to be. Just because Google missed the AI train doesn't mean wishful thinking the market to be killed by FOSS is going to make it so. As usual with those things it's impossible to know in advance what's going to happen, but odds are not disfavoring chatgpt as much as this article says.

bickfordb 4y ago

A good example of this is Youtube

freediver 4y ago

> AI text generation competitors like Cohere and Anthropic will never be able to compete with Microsoft/Google/Amazon on marginal cost. Anthropic already does, with its models. They are same price or cheaper than OpenAI, with comparable quality. > Having enough scale to perpetually offer free/low-cost compute is a moat. Rather than a moat it is a growth strategy. At one point in time you need to start to monetize and this is the moment when rubber hits the road. If you can survive monetization and continue to grow, now you have a moat.