3 ms·
Lest we forget that this stream-of-consciousness confusion was state of the art just a few years ago. It makes sense if you think about it: a small model's "in
by nerdponx 2y ago
Lest we forget that this stream-of-consciousness confusion was state of the art just a few years ago.
It makes sense if you think about it: a small model's "internal state" isn't rich enough to keep track of whatever it was supposed to be talking about.
It makes me think that the reason LLMs need to be so large is that the internal state needs to be bigger than a typical human "idea", whatever that might mean.
- acchow 2y agoThe way we do LLMs now is that the program and the data are one and the same. The program mutates itself as it "executes". This is probably also how the brain works since there is no hard separation between "memory" neurons and "data processing" neurons. (biology has no hard separation in general).