5 ms·
Yesterday, while using ChatGPT-4, it gave me a very long answer almost instantly. It felt like I was using ChatGPT-3.5, including the poor quality of the answer
by haolez 3y ago
Yesterday, while using ChatGPT-4, it gave me a very long answer almost instantly. It felt like I was using ChatGPT-3.5, including the poor quality of the answer. In the following prompts, it became slow again, as GPT-4 is supposed to be. The quality improved as well.
I think they are trying some aggressive customization on their infra to try to make it economically viable, but it's just speculation at this point.
- andai 3y agoI heard GPT-4 described as "eight GPT-3's in a trenchcoat" but I'm not sure how accurate that is.
- brandall10 3y agoSeveral people hinted/remarked (starting w/ George Hotz, then others more closely linked to OpenAI) that it's a Mixture of Experts* approach comprised of 8 220B parameter models. * https://arxiv.org/pdf/2101.03961.pdf https://arxiv.org/pdf/2101.03961.pdf
- j45 3y agoI wonder if additional layers of factories of factories approach can continue to improve it. I'm not familiar enough with the technology but could it be possible to create a prompt, or multiple prompts to stitch together 8 similtaneous calls to GPT3.5 pulled together and see if the quality is similar to GPT4?
- andai 3y agoI think the experts part implies that each "GPT-3" has been specialized in some way?
- brandall10 3y agoThey're not 3.5 models, which are 175B param, they're 220B and the idea is the fine tuning is different for each which is where the 'expert' comes in.
- j45 3y agoAh, that makes sense, thanks for clarifying
- ttul 3y agoThe mixture of experts approach was found to be inferior than a straightforward transformer. However, researchers discovered that MoE models give substantially better results when guided by fine-tuning. My guess is that GPT-4 was an experiment to prove out that theory at huge scale and I also suspect that its performance astonished OpenAI as much as everyone else.
- s1gnp0st 3y agoIf it was an experiment I'd like to know why I'm still paying for access to something claiming to be it.
- nicky0 3y agoBecause you are getting value from it. Why else would you pay?
- babushkanazi 3y ago[dead]
- insanitybit 3y agoBecause you signed up to pay for a beta service???? You can stop paying at any time.
- JohnFen 3y agoIf you're paying for it, it's a released product, not a beta. No matter how much a company wants to claim it's a "beta", it's simply not. The company just wants to be able to adhere to lower standards. OpenAI is hardly the only company that pulls these shenanigans, too.
- insanitybit 3y agoNo, beta has nothing to do with if it's paid or not. Lots of products are released, and you pay for them, with early access/ beta. No one lied to you. No one tricked you. It is in beta. You chose to pay for it.
- rogerkirkness 3y agoIt's actually 16 GPT 3.5s in a trenchcoat where each one is slightly different, like the minions.
- andai 3y agoCan anyone elaborate on the specialization? Does it have to do with segmenting the dataset, or with training on different kinds of tasks?
- rogerkirkness 3y agoFine tuning to a specific more constrained set of tasks is my understanding.
- idkyall 3y agoI wonder if maybe they cache and re-serve answers to similar prompts? Probably they could use a GPT-4 response as a template and have GPT-3.5 tweak it to be specific to your prompt
- chaxor 3y agoIf their statement is that they haven't changed the weights, there still an immense number of things they can toy with. For example, the prompt may add more 'safety' language in it, which can cause strange differences to occur, or typical_p sampling values, top_p, top_k, etc, or if they do use mixture of experts, they may even be able to 'use less experts' and only run 1/4 of the models, such that the speed is improved greatly. There's plenty of ways to make the model change output without retraining.
- rich_sasha 3y agoMy personal suspicion is that there is some kind of grid/beam search at the very end. You can kind of see (imagine?) it as output speed is not constant. So you crank down the iterations and voila. Same weights, poorer output.
- dumdumchan 3y agoNo surprises here. The entire architecture is based on mass production. There is an immutable base model and the only mutable part is the chat context. There is fine tuning but it takes a long time and there is no temporal relation between the fine tuning prompts. Even then it costs a ton of money. This immutable model is shared among all the users of their services. There has to be a supercomputer version of this product that focuses on quality over quantity.