4 ms·
> LLM's don't actually understand anything (as greater concepts), but rather operate as complex probability machines? What does it mean to "operate as a probab
by thomasahle 3y ago
> LLM's don't actually understand anything (as greater concepts), but rather operate as complex probability machines?
What does it mean to "operate as a probability machine"? And what does it mean to understand anything?
One recent example of understanding is that llms/transformers learn to parse context free grammars via dynamic programming (https://arxiv.org/abs/2305.02386 https://arxiv.org/abs/2305.02386). Basically they've understood what's going on well enough to mold their neurons I to the optimal algorthm for parsing this kind of text.
I think they understand lots of things like this. Of course there's other things they don't understand or just pretend to understand.