4 ms·

This is the real risk to OpenAI's business model. If it turns out that you can get most of the same outcome with drastically smaller and cheaper models, then O

This is the real risk to OpenAI's business model. If it turns out that you can get most of the same outcome with drastically smaller and cheaper models, then OpenAI is going to have a hell of a time keeping customers around as it will just be a race to the bottom on price and bigger, more expensive models will lose just from a hardware cost standpoint.

rashkov 4y ago
What about the high quality training data that OpenAI has encoded into ChatGPT? Do these other models come close to that?
- woeirua 4y ago
  Why couldn't you just use OpenAI's API to feed prompts and then take the outputs and use them to train your own model to exfiltrate the best features of GPT?
  - wsgeorge 4y ago
    Because it would be against their TOS, and things could look ugly, legally.
    - AJ007 4y ago
      It's still an open question if any of these models, trained on copyright work, will themselves be eligible for copyright protection.
    - ImprobableTruth 4y ago
      Is this a bit? If it's illegal to train on copyrighted material, then OAI has broken the law ten times over by training GPT3. There's absolutely zero reason for them to sue, they'll just ban the responsible people.
    - nickthegreek 4y ago
      I think their TOS forbids using the API for this. I don't think it covers the use of the web interface.
      - circuit10 4y ago
        However: "You may not [...] except as permitted through the API, use any automated or programmatic method to extract data or output from the Services, including scraping, web harvesting, or web data extraction;"
        nickthegreek 4y ago
        Can’t be automated, so manual extraction is allowed.
    - ImHereToVote 4y ago
      Ironic
    - tspike 4y ago
      How many TOS agreements do you suppose they violated while training their models?
  - xpe 4y ago
    Give it a try if you feel like it is a good thing to do. I'm sure some nation states are doing it. P.S. this comment does not reflect my personal values. But I would rather someone with values try it almost like a white hat pen test.
- typon 4y ago
  That's how Alpaca is made
smoldesu 4y ago
On the flip-side, OpenAI is primed to destroy their competitors. Partnership with Microsoft means they can buy Azure compute at-cost if need be. Their current portfolio of models is diverse on the expensive and cheap ends of the spectrum, with thousands of people on Twitter and HN still giving them lip-service. With dozens of clones hitting the market, OpenAI is the only one staying consistently relevant. The widespread adoption of local AI won't obsolete a well-priced AI API. I feel like we learned that lesson pretty thoroughly in the SaaS era.
- xpe 4y ago
  > The widespread adoption of local AI won't obsolete a well-priced AI API. I feel like we learned that lesson pretty thoroughly in the SaaS era. Unless I am misunderstanding (?), this seems like an overgeneralized lesson. There are many key differences between these situations that make such a connection unlikely. Could you explain your reasoning?
- ijustlovemath 4y ago
  The difference between this and SaaS is that businesses have been moving their (end user) products to SaaS due to wider broadband availability, as well as greed (read: MRR), but on the LLM side, people are building new products with it, so the incentives are to keep your costs low (or free) so you can make more money once you release.
xpe 4y ago
No disrespect to the author intended, but the above comment is muddled. 1. OpenAI, the organization, is not equivalent to its chat offering. 2. Saying "the" real risk isn't persuasive. Let's examine many risks before claiming one is the most significant. Also, "real" is this usage often a throwaway (i.e. unneeded) word, in editor speak. 3. Let's talk about OpenAI's "business model" (though such discussions are tricky). 3A. Originally, OpenAI wasn't trying to "hold onto" AI advancements. It claimed to be a broadly funded way to explore fundamental questions of artificial intelligence in a non-commercial, ethical way. 3B. Of course, the above claim was largely aspirational, because it wasn't baked into their DNA in way that could survive the surrounding temptations for more funding, glory, and resources. 3C. Even with their more commercialized model of the last several years, it seems their business model feels like (a) fundraise in exchange for (b) (claimed) collective good open source, tools and shared research. 3D. OpenAI feels to me more and more like a commercial research lab; there does seem to be a lot of commercial partnering with their funding organizations (e.g. Microsoft). 4. I doubt the leadership there views the current ChatGPT models as unchanging. I expect there is a considerable revenue stream around the space. OpenAI is well positioned to play the game several steps ahead of others. I would frame the broader question this way: for many years, there has been a hunger for this deeper AI research, due not only to (i) the expertise and resources required, but also (ii) to this hope that there is an organization that can maybe keep it within human or ethical bounds. Unfortunately, this amorphous hope doesn't seem to be matching the actual organizational incentives nor dynamics. It is also unclear how much demand the public in free market will have for nobler research. My position on these kinds of things is simple: follow the money. If we want an accountable public interest, AI research laboratory it's going to have to be designed, funded, and overseen very differently.
nico 4y ago
That’s why they are moving so fast and trying to get as much press/media attention as possible. They want to stay top of mind. Think about CocaCola, anyone can make a drink just as good. But it’s almost impossible to build their brand and distribution from scratch.
lfciv 4y ago
I wouldn't underestimate the power of momentum

rashkov 4y ago

What about the high quality training data that OpenAI has encoded into ChatGPT? Do these other models come close to that?

woeirua 4y ago
Why couldn't you just use OpenAI's API to feed prompts and then take the outputs and use them to train your own model to exfiltrate the best features of GPT?
- wsgeorge 4y ago
  Because it would be against their TOS, and things could look ugly, legally.
  - AJ007 4y ago
    It's still an open question if any of these models, trained on copyright work, will themselves be eligible for copyright protection.
  - ImprobableTruth 4y ago
    Is this a bit? If it's illegal to train on copyrighted material, then OAI has broken the law ten times over by training GPT3. There's absolutely zero reason for them to sue, they'll just ban the responsible people.
  - nickthegreek 4y ago
    I think their TOS forbids using the API for this. I don't think it covers the use of the web interface.
    - circuit10 4y ago
      However: "You may not [...] except as permitted through the API, use any automated or programmatic method to extract data or output from the Services, including scraping, web harvesting, or web data extraction;"
      - nickthegreek 4y ago
        Can’t be automated, so manual extraction is allowed.
  - ImHereToVote 4y ago
    Ironic
  - tspike 4y ago
    How many TOS agreements do you suppose they violated while training their models?
- xpe 4y ago
  Give it a try if you feel like it is a good thing to do. I'm sure some nation states are doing it. P.S. this comment does not reflect my personal values. But I would rather someone with values try it almost like a white hat pen test.
typon 4y ago
That's how Alpaca is made

smoldesu 4y ago

On the flip-side, OpenAI is primed to destroy their competitors. Partnership with Microsoft means they can buy Azure compute at-cost if need be. Their current portfolio of models is diverse on the expensive and cheap ends of the spectrum, with thousands of people on Twitter and HN still giving them lip-service. With dozens of clones hitting the market, OpenAI is the only one staying consistently relevant. The widespread adoption of local AI won't obsolete a well-priced AI API. I feel like we learned that lesson pretty thoroughly in the SaaS era.

xpe 4y ago
> The widespread adoption of local AI won't obsolete a well-priced AI API. I feel like we learned that lesson pretty thoroughly in the SaaS era. Unless I am misunderstanding (?), this seems like an overgeneralized lesson. There are many key differences between these situations that make such a connection unlikely. Could you explain your reasoning?
ijustlovemath 4y ago
The difference between this and SaaS is that businesses have been moving their (end user) products to SaaS due to wider broadband availability, as well as greed (read: MRR), but on the LLM side, people are building new products with it, so the incentives are to keep your costs low (or free) so you can make more money once you release.

xpe 4y ago

No disrespect to the author intended, but the above comment is muddled. 1. OpenAI, the organization, is not equivalent to its chat offering. 2. Saying "the" real risk isn't persuasive. Let's examine many risks before claiming one is the most significant. Also, "real" is this usage often a throwaway (i.e. unneeded) word, in editor speak. 3. Let's talk about OpenAI's "business model" (though such discussions are tricky). 3A. Originally, OpenAI wasn't trying to "hold onto" AI advancements. It claimed to be a broadly funded way to explore fundamental questions of artificial intelligence in a non-commercial, ethical way. 3B. Of course, the above claim was largely aspirational, because it wasn't baked into their DNA in way that could survive the surrounding temptations for more funding, glory, and resources. 3C. Even with their more commercialized model of the last several years, it seems their business model feels like (a) fundraise in exchange for (b) (claimed) collective good open source, tools and shared research. 3D. OpenAI feels to me more and more like a commercial research lab; there does seem to be a lot of commercial partnering with their funding organizations (e.g. Microsoft). 4. I doubt the leadership there views the current ChatGPT models as unchanging. I expect there is a considerable revenue stream around the space. OpenAI is well positioned to play the game several steps ahead of others. I would frame the broader question this way: for many years, there has been a hunger for this deeper AI research, due not only to (i) the expertise and resources required, but also (ii) to this hope that there is an organization that can maybe keep it within human or ethical bounds. Unfortunately, this amorphous hope doesn't seem to be matching the actual organizational incentives nor dynamics. It is also unclear how much demand the public in free market will have for nobler research. My position on these kinds of things is simple: follow the money. If we want an accountable public interest, AI research laboratory it's going to have to be designed, funded, and overseen very differently.

nico 4y ago

That’s why they are moving so fast and trying to get as much press/media attention as possible. They want to stay top of mind. Think about CocaCola, anyone can make a drink just as good. But it’s almost impossible to build their brand and distribution from scratch.

lfciv 4y ago

I wouldn't underestimate the power of momentum