4 ms·
Are these trained on internal Code bases or just the public repositories?
by ramshanker 3y ago
Are these trained on internal Code bases or just the public repositories?
- albertzeyer 3y agoWithout knowing any details, I'm almost sure that they did not train it on internal code, as it might be possible to reveal that code otherwise (given the right prompt, or just looking at the weights).
- make3 3y agopeople are able to extract training data from llms with different methods, so there's no way this was trained on internal code
- deleted 3y ago[deleted]
- eigenvalue 3y agoWould be a really bad idea to train on internal code I would think. Besides, there is no shortage of open source code (even open source created by Meta) out there.
- changoplatanero 3y agoCorrect that it’s a bad idea to train on internal code. However surprisingly there is a shortage of open source code. These models are trained on substantially all the available open source code that these companies can get their hands on.
- andy99 3y agoThe github [0] hasn't been fully updated, but it links to a paper [1] that describes how the smaller code llama models were trained. It would be a good guess that this model is similar. [0] https://github.com/facebookresearch/codellama https://github.com/facebookresearch/codellama [1] https://ai.meta.com/research/publications/code-llama-open-foundation-models-for-code/ https://ai.meta.com/research/publications/code-llama-open-fo...