3 ms·
>We study a frontier LLM in its purest inference regime - a single generation pass over a prompt containing the full training and test data, with no tools, no a
by heaney-555 2mo ago
>We study a frontier LLM in its purest inference regime - a single generation pass over a prompt containing the full training and test data, with no tools, no agentic scaffolding, and no fine-tuning
Sigh. So this is somewhat interesting niche academic research but utterly irrelevant to real-world use cases.
- bonsai_spool 2mo agoAnd written by an LLM, to boot.
- keremturhan 2mo ago[flagged]
- scott_s 2mo agoI find that an odd take. The paper claims to establish what causes the problem: dimensionality. They are clear in that they don't understand why. But this sort of work is what needs to be done to eventually solve the problem.
- JSR_FDED 2mo ago[dead]
- gowld 2mo agoSolve what problem? My hammer can't drive screws. Is that a problem to solve?
- Jensson 2mo agoIf you market your hammer as an all purpose tool, yes. If LLM are worse than humans at predicting data it is valid to wonder why, you could think they would be better at it since they are next token predictors.
- antonvs 2mo agoThe problem it helps solve is our lack of understanding of how the models work. The way we got these models in the first place is by people researching to try to understand how the relevant systems worked. There’s a great deal we don’t know about how LLMs work. This paper is exploring one aspect of the limitations of LLMs, which helps to understand them better - their capabilities and limitations, and the underlying causes. This kind of work is exactly what’s needed to develop better models - not just LLMs.
- mwkaufma 2mo agoSo the best way to run a controlled trial is to maximize the external noise and confounding variables?