3 ms·
So think about it. If Morgan Stanley is right and OpenAI is currently training GPT-5 for hundreds of millions of dollars, they might not make it directly availa
by ftxbro 4y ago
So think about it. If Morgan Stanley is right and OpenAI is currently training GPT-5 for hundreds of millions of dollars, they might not make it directly available. They could use Alpaca power to use it to boost GPT-4 or GPT-3. This way they could have these super optimized models that can fit in smaller memory and compute their tokens faster than any of the competition. This would be the opposite of OpenAI having a problem. It would be this huge economic moat that lasts for a while until more efficient hardware and more capital spending by their competition degrades it. It would even solve the problem of super human GPT-5 cognition. That would be moot if it's never released and only used to Alpaca-ize their own smaller models.
- atleastoptimal 4y agoThat's true. Anything open source to the public is obviously open source to a company. They can create public, miniaturized versions of their models and release capabilities only when it suits them, using Alpaca style training to minimize costs for their API.
- meghan_rain 4y agoSource on Morgan Stanley talking about GPT-5?
- ftxbro 4y agoI hope my analysis makes sense regardless of when or how OpenAI ends up training GPT-5, but the Morgan Stanley thing is an internet rumor for example https://news.ycombinator.com/item?id=35222180 https://news.ycombinator.com/item?id=35222180
- meghan_rain 4y agoThanks
- huijzer 4y agoEven without Morgan Stanley claims, Greg Brockman and Satya have both said that GPT-3.5 was already internally used by them since the summer of last year and GPT-4 in December. Greg mentions it in https://youtu.be/YtJEfTTD_Y4 https://youtu.be/YtJEfTTD_Y4 for example
- light_hue_1 4y ago> It would even solve the problem of super human GPT-5 cognition. That would be moot if it's never released and only used to Alpaca-ize their own smaller models. They would not have a moat from this strategy. At the end of the day, they still make available a model with high performance. The whole point is that anyone, for very little money, can fine-tune another model to match the performance of OpenAI's models.
- jostiniane 4y agoThat's a valid perspective IMO but corporations are too slow to react and keep up.. Another thing is, if models that are good enough for the general use case that I can fit in my medium range hardware then I definitely don't need to pay for the more perfect one.. A couple of such good enough models are already there and a lot of us are already not following or waiting for the GPT-10.. That's the point of the article and why OpenAI are visibly insecure about those models.