2 ms·
i feel you would recognise that more as a quirk of how humans think, remember that LLMs think fundamentally differently to you and i. i would be curious about s
by ambichook 3y ago
i feel you would recognise that more as a quirk of how humans think, remember that LLMs think fundamentally differently to you and i. i would be curious about someone making a benchmark like that and using it to compare as an experiment however
- foota 3y agoI'm not trying to anthropomorphize the model, but it's not hard to imagine that a model would attribute significance to something completely out of context, and hence "focus" on it when computing attention. Another possible synthetic benchmark would be to present a list of key value pairs and then ask it for the value corresponding to different keys. Or present a long list of distinct facts and then ask it about them. This latter one could probably be sourced from something like a trivia question and answers data set. I bet there's something like that from Jeopardy.