3 ms·
This is unsubstantiated. The only folks who know exactly how GPT-4 works are employed at OpenAI. The rest of us can only guess.
by npsomaratna 3y ago
This is unsubstantiated. The only folks who know exactly how GPT-4 works are employed at OpenAI. The rest of us can only guess.
- YetAnotherNick 3y agoEven if I just go with Sam Altman's public comment, I would have came to similar conclusion: GPT-4 is big and it is hard to make it is faster. The secret sauce and moat lies in data though. I have heard rumour that they have paid competitive coders to write and annotate code with information like complexity for them.
- astrange 3y agoGPT4 can diagram sentences using link grammar parsing (https://www.link.cs.cmu.edu/link/ https://www.link.cs.cmu.edu/link/) which is obscure enough I really don't think they've generated data for it. So it can get pretty good without that.
- YetAnotherNick 3y agoIt's obvious they use data from github and other places. I am talking about extra 0.00..1% very high quality data they (likely)created.