4 ms·
That's true, I've noticed this with GPT-4. It'll ignore some part of its context, and when I point it out, it knows, so it's clearly still in its context, but i
by 3cats-in-a-coat 3y ago
That's true, I've noticed this with GPT-4. It'll ignore some part of its context, and when I point it out, it knows, so it's clearly still in its context, but it didn't know it has to look it up for a particular answer.
We also have the same problem with memory so I empathize.
- behnamoh 3y agoThis limitation of LLMs actually makes me think more of them like our own brains. The similarities between transformers and brain are interesting.
- 3cats-in-a-coat 3y agoDiffusion also. It produces startlingly dream-like imagery, especially the videos. Things like that to me are a solid evidence that we "nailed it" with reproducing the neural activity of a living being. Of course, architectures will come and go, but they just nudge the shape of the network in a given direction, they don't define it, as most of the network is trained, evolved. It has hidden architecture no human has designed, and the patterns in that hidden architecture resemble our own thought patterns.