4 ms·
OpenAI, through Microsoft + Github, has access to unfathomable amounts of source code training data and would be just fine without StackOverflow.
by xd1936 1y ago
OpenAI, through Microsoft + Github, has access to unfathomable amounts of source code training data and would be just fine without StackOverflow.
- PaulDavisThe1st 1y agoSO provided the connection between natural language (primarily English) and source code. Access to source code alone doesn't do that, commented code nothwithstanding.
- CamperBob2 1y agoNo, compsci textbooks and language manuals do that. SO is not the primary, canonical educational resource you seem to think it is, and they'd be the first to agree.
- ceejayoz 1y ago> No, compsci textbooks do that. No, they don't.
- CamperBob2 1y agoSigh. Yes, you're right, programming textbooks do that. Now, where's my cilice...
- kazinator 1y agoBy and large compsi text books are not sources of large amounts of working code in a specific language. Some programming-oriented ones may be; does Numerical Recipes in C count as a comp sci book?
- CamperBob2 1y agoTrue, I was assuming that people would think a bit more abstractly, or at least a bit more generously, but sometimes I forget where I am. By "compsci" I mean everything from graduate-level theoretical texts all the way down to "101 BASIC Programs for the TRS-80." In the old days, magazine articles would also present practical code alongside plaintext explanations of how it worked. There's still no shortage of tutorial content, although not as much in paper form, and even less on Stack Overflow.
- kazinator 1y agoI don't suspect that SO alone is anywhere nearly sufficient to train LLMs to predict solutions to coding problems and write code. There must be additional training going on with tagged sets. I've heard about people being employed by AI companies to solve programming problems just for the sake of generating training pairs.