2 ms·
I know how language models work. It's funny how you paint this obvious in retrospect. Guess Jan Leike doesn't know how statistical language models work. https
by famouswaffles 4y ago
I know how language models work. It's funny how you paint this obvious in retrospect.
Guess Jan Leike doesn't know how statistical language models work.
https://mobile.twitter.com/janleike/status/1625207251630960640?t=Z3RCi1T9TFAY9ylf8Y0vXA&s=19 https://mobile.twitter.com/janleike/status/16252072516309606...
- PheonixPharts 4y agoJan Leike's point is a bit more subtle there. He's not surprised it can learn translation with a relatively small number of training examples, but surprised that it seems to know some instruction with no examples. And just a few Tweets later speculates it's likely because there are examples in the training data, and the success of the model to follow instructions in a non-English language from "nothing" is proportional to the presence of those examples in the training data [0]. I don't think he would disagree with anything in my statement. 0. https://mobile.twitter.com/janleike/status/1625215091988926464 https://mobile.twitter.com/janleike/status/16252150919889264...
- famouswaffles 4y agoWell sure, instruction following was my second point.
- famouswaffles 4y agoAnd my last point was code models performing better on MMLU and the like. How'd you explain that one ?