2 ms·
I find that an odd take. The paper claims to establish what causes the problem: dimensionality. They are clear in that they don't understand why. But this sort
by scott_s 2mo ago
I find that an odd take. The paper claims to establish what causes the problem: dimensionality. They are clear in that they don't understand why. But this sort of work is what needs to be done to eventually solve the problem.
- JSR_FDED 2mo ago[dead]
- gowld 2mo agoSolve what problem? My hammer can't drive screws. Is that a problem to solve?
- Jensson 2mo agoIf you market your hammer as an all purpose tool, yes. If LLM are worse than humans at predicting data it is valid to wonder why, you could think they would be better at it since they are next token predictors.
- antonvs 2mo agoThe problem it helps solve is our lack of understanding of how the models work. The way we got these models in the first place is by people researching to try to understand how the relevant systems worked. There’s a great deal we don’t know about how LLMs work. This paper is exploring one aspect of the limitations of LLMs, which helps to understand them better - their capabilities and limitations, and the underlying causes. This kind of work is exactly what’s needed to develop better models - not just LLMs.