sichengo

4 ms·

1. 1.▲
   Why Do LLMs Hallucinate?(sicheng.dev)

   3 points by sichengo 8mo ago|2 comments
2. 2.▲
   by sichengo 8mo ago

   Yes, I do admit that I simplified the mechanism in the article, but my question is why the scale of next-token prediction yields reasoning-like behavior.
3. 3.▲
   by sichengo 8mo ago

   Hmm, good point, but google only retrieves information from the web but LLMs do generate a new continuations from learned distributions. I would say the interesting question is that why modeling reasoning traces at scale produces reasoning
4. 4.▲
   If LLMs Only Predict the Next Token, Why Do They Work?(sicheng.dev)

   3 points by sichengo 8mo ago|7 comments