3 ms·
> so clearly an LLM that does math well only does so by ignoring the majority of the space it is trained on There are probably good reasons why LLMs are not th
by grogers 9mo ago
> so clearly an LLM that does math well only does so by ignoring the majority of the space it is trained on
There are probably good reasons why LLMs are not the "ultimate solution", but this argument seems wrong. Humans have to ignore the majority of their "training dataset" in tons of situations, and we seem to do it just fine.
- D-Machine 9mo agoIt isn't wrong, just think about how weights are updated via (mini-)batches, and how tokenization works, and you will understand that LLM's can't ignore poisoning / outliers like humans do. This would be a classic recent example (https://arxiv.org/abs/2510.07192 https://arxiv.org/abs/2510.07192): IMO because the standard (non-robust) loss functions allow for anchor points .