3 ms·
My take is when the model can't do the things that you would expect it to do if it were reasoning, and instead does the things that you would expect it to do if
by sgt101 3y ago
My take is when the model can't do the things that you would expect it to do if it were reasoning, and instead does the things that you would expect it to do if it were an approximate retrieval system with an enormous knowledge base.
this is a good paper: https://arxiv.org/abs/2309.12288 https://arxiv.org/abs/2309.12288
- famouswaffles 3y agoThe "reversal curse" has nothing todo with inference or the reasoning abilities of a pre-trained model. GPT-4 or whatever has no problem telling you the right deduction with information presented in the context. This is a memory retrieval from training issue and is completely unsurprising.