5 ms·
This just shows how great of an acquisition GitHub was for Microsoft. Having access to all that code is a real differentiator for training. I've heard that cod
by losvedir 3y ago
This just shows how great of an acquisition GitHub was for Microsoft. Having access to all that code is a real differentiator for training.
I've heard that code helps models more generally, with reasoning and language, because it's clearly structured and things like conditionals and control flow apply to logic questions and making deductions and inferences in just plain conversation. It seems plausible to me, but I haven't seen anything formal about the idea.
- 999900000999 3y agoTechnically I think anything MIT licensed can be cloned and trained on. At the same time Microsoft is under no obligation to provide an easy way for language learning model to do it, given that Microsoft is hosting all of this content they can easily have co-pilot learn from every bit of code that's hosted on GitHub with a permissible license.