3 ms·
I wish he would make a blog post about his plans then. Biggest question for me is, when is the next version after gpt-4-0314 coming out? Also it would be good
by ilaksh 3y ago
I wish he would make a blog post about his plans then.
Biggest question for me is, when is the next version after gpt-4-0314 coming out?
Also it would be good for people to know the real situation with GPUs or AI accelerators that can handle their LLMs.
Such as, did they order them and are now literally waiting for them to be manufactured, or are they installing them in a datacenter? Any kind of peek into those operations might help people understand the challenges so they complain less.
Also this is tangential but is there a factory and design center in North America that can create and manufacture the necessary level of AI hardware? Or does OpenAI's ability to take on new customers down the line depend on Taiwan and holding off WWIII?
I guess a lot of things kind of depend on that. But I don't see any good plans on how to avoid it.
- az226 3y agoIt’s already out. It’s worse but faster at generating tokens.
- ilaksh 3y agoI don't see any announcement. You mean they just silently replaced gpt-4-0314 somehow? I think that if they changed something its actually not the model, but maybe something that affects the performance of the model.
- az226 3y agoCorrect. If you go to the playground, choose Chat, you see two options, GPT4 and GPT4 (3/14). The former appears to be powering CharGPT4 right now and the latter is what used to be powering it.
- whatyesaid 3y agoPretty sure he said a few months ago that scaling further than GPT 4 isn't going to help enough, and that other breakthroughs are needed or something about building on top of GPT 4. Since then the only foundational breakthrough was much bigger context windows. Obviously to even run and train GPT 4 now is too compute intensive, though yeah.
- ahefner 3y agoWere you limited by GPU capacity, would you intentionally downplay the potential benefit to further scaling the model so as to discourage competitors (e.g., Google) who would be less GPU-limited from pursuing such a direction themselves? I don't necessarily think this is the case, but it occurs to me that were there any practical impediment (however emphemeral) to them commercially exploiting a larger scale model, they'd likely adopt a posture of dismissing further scaling until they were ready.
- whatyesaid 3y agoI heard that ChatGPT 4 runs in the red even with $20 subscription. Makes sense since only 25 messages every 4 hours. The nerf coincides with browsing taking more power up probably. Google probably doesn't want people using GPT 6 tech instead of search entirely, which is a real possibility since Google is often filled with SEO spam nowadays. I think they just want LLM + integrated with ads. Google had better results like 15 years ago for general information. Of course their nerfs are all intentional. When compute and techniques improve/scale they can unnerf. If they catch wind of a competing product they can also unnerf. They still have stuff developed they can't affordably deploy like multimodal, it can already take images.
- Spivak 3y agoI just wish they would give access to whatever text model is backing gpt4 like text-davinci-004. It's really nice to have a model that isn't weirdly trying to be conversational all the time when you want to do things over than have conversations.
- ilaksh 3y agoI think the issue is that both the alignment stuff and performance in terms of reasoning is largely the result of the reinforcement learning that happened after that model was completed.