3 ms·
It's possible that LLMs build ASTs internally for programming. I have no 1st hand data on this, but it would not surprise me at all.
by kenjackson 1y ago
It's possible that LLMs build ASTs internally for programming. I have no 1st hand data on this, but it would not surprise me at all.
- astrange 1y agoLLMs don't have memory, so they can't build anything. Insofar as they produce correct results, they have implicit structures corresponding to ASTs built into their networks during training time.
- kenjackson 1y ago"LLMs don't have memory" That's interesting. Is there research into adding memory or has it been proven that it provides no pragmatic value over any context it outputs?