3 ms·
Broadly I agree with your position, but: > If you can find a way to make the context window on the scale of the human brain, you may be able to mostly mitigate
by ben_w 2y ago
Broadly I agree with your position, but:
> If you can find a way to make the context window on the scale of the human brain, you may be able to mostly mitigate this.
Human brains have a much smaller context window than AI do. We can't pay attention to the last 128,000 concepts that filtered past our sensory systems — our conscious considerations are for about seven things.
There's a lot of stuff that we don't yet understand well enough to reproduce with AI, but context length is the wrong criticism for these models.
- rglover 2y ago> context length is the wrong criticism for these models You're right. What I'm getting at is the overall speed, efficiency, and accuracy of the storage, retrieval, and processing capability of the human brain.
- wkat4242 2y agoIt's kinda crazy that it can run on a few slices of bread when LLMs need kilowatts of power to write a simple paragraph :)