4 ms·
You’re assuming the bulk of ChatGPT’s coding knowledge comes from stack overflow. I suspect training directly on code bases and documentation is more important.
by valine 3y ago
You’re assuming the bulk of ChatGPT’s coding knowledge comes from stack overflow. I suspect training directly on code bases and documentation is more important.
- Gollapalli 3y agoNo, ChatGPT isn’t AI in that sense. It can fire back documentation at you, but a lot of what it does is just look at the word it’s spit out, and pick the next word or snippet that looks right. Same thing with code. It just spits out what it’s trained to interpolate the next snippet to be. Anything involving genuine reasoning needs to be done by a human. It’s a great documentation parser. A step up in search engines. But a thinking machine it is not (yet).
- valine 3y agoIf this were true ChatGPT would be useless for any topic that doesn't have a vast Q&A dataset. The purpose of instruct fine tuning is to illicit the model to output in a question answer format. The bulk of the model's knowledge comes from the unstructured dataset. I'm also not sure how "reasoning" at inference time is relevant here. You seem to be conflating unsupervised offline training with inference. The ironic thing is that the behavior you describe actually works, you can try it right now. Add code documentation to your context window and ChatGPT will happily write code using that documentation as reference.
- gostsamo 3y agonot every product has good documentation nor the code is self explanatory. SO fills in the gap between a requirement and its translation in code. Only OpenAI can say for sure how important is SO for them, but I doubt that it is insignificant.