3 ms·
Now I wonder what would happen if we finetuned GPT-3 on stackoverflow. Would it become better at code ? Good enough to assist any beginner in learning ? That w
by nephanth 4y ago
Now I wonder what would happen if we finetuned GPT-3 on stackoverflow. Would it become better at code ? Good enough to assist any beginner in learning ?
That would be huge
- goodside 4y agoThis is exactly the idea behind GPT-3 Codex, but using GitHub instead of SO. It does, indeed, become much better at programming than the baseline model.
- joshspankit 4y agoOne of the meta-challenges as I see it is “if we had a data set comprised of only the best information from the best experts we could make an AI that beats them all”. Because I think that’s both true and entirely unrealistic. Like, how would you quantify who the best experts are, and how would you verify that you’re not training on wrong data from those best experts? Maybe they had an off day or a specific agenda. Even browsing through verified SO answers there are still answers that are outdated and answers that are just plain wrong. The greater the experience needed to answer a specific question, the more unlikely it is that there is a clear and correct answer given.
- mmahemoff 4y agoSO is far from perfect, but of all the websites out there, SO has strong signal in this regard thanks to the moderation and metadata available.
- IshKebab 4y agoI'm sure the GPT-3 training set includes Stackoverflow already. Would be weird if it didn't.
- taneq 4y agoJust imagine being able to hit ctrl+space in your editor to add "this question has been marked as a duplicate." :P