2 ms·
Well, I guess it's just this kind of talk it found in its training data? They say 'zero (human) data', but in fact they start with an entire language model tha
by eru 2y ago
Well, I guess it's just this kind of talk it found in its training data?
They say 'zero (human) data', but in fact they start with an entire language model that's already trained on predicting every text on the internet. There's plenty of people writing about obfuscated code on there.
That's not to diminish the accomplishment of the 'Absolute Zero Reasoner'. It's just a bit more nuanced than 'zero data'. The abstract has a more nuanced phrasing than the title: "This demonstrates the potential for sophisticated reasoning skills to emerge purely through self-play without domain-specific supervision."