5 ms·
Part of the evaluation procedures for the models like these is checking whether the output they produce is present in the training set. The fraction of samples
by the8472 4y ago
Part of the evaluation procedures for the models like these is checking whether the output they produce is present in the training set. The fraction of samples being verbatim reurgitations is low unless you prompt-engineer your way to that, copilot being able to recite the fast inverse square root function being a common example. But the ability to recite some samples is not proof of inability to create novel outputs.
- benreesman 4y agoOh sure. I didn’t mean to imply that GPT-6 will have memorized Wikipedia. As you start to get to the size and comprehensiveness in the corpus that’s going on now, novel model outputs approach being something that might not have been in the training set, but likely will be in the future.
- trention 4y ago>The fraction of samples being verbatim reurgitations is low Copilot spitting out a function out of its training data with changed variable names to match those in your file(s) - and no one is actually testing what proportions of results those are - is still regurgitation.
- the8472 4y ago> and no one is actually testing what proportions of results those are The idea that models could possibly overfit the training data is hardly a new idea. It's standard practice to test for that. Check section 7 of the PaLM paper for example. https://arxiv.org/abs/2204.02311 https://arxiv.org/abs/2204.02311
- benreesman 4y agoCopilot is weird. The N in the NLP training set and arch implies a fuzzy match. Why anyone would rather start with a plausible but broken-in-a-subtle way buffer rather than a blank one is beyond me.