3 ms·
Interestingly, Code models are constrained even more by difficulties of tokenization in light of - most crucially - us not having actually that much code to tra
by zwaps 4y ago
Interestingly, Code models are constrained even more by difficulties of tokenization in light of - most crucially - us not having actually that much code to train on (we already train on all of of github, and it doesn't "saturate" the model).
At this stage, we are back to improving model efficiency, I think, especially for code models. But not there yet.
Sorry for the rambling, the actual answer is no I do not have a really good codex type model in open source.. yet
- boredemployee 4y agoI see. The Open AI code generator gave me really impressive results for basic to intermediate questions in the data analytics space. I think it's a function of the context you give about the problem (aka what are the literal meaning of the columns in the business context) and how objective your question - to the model - is, plus some other internal model variable that I'm completely unaware of. But it's nice to have your input so I can understand a little bit what happens under the hood!