3 ms·
> And a final note, predicting one token at a time is what we are doing as humans when we speak... I wouldn't be surprised if we could predict token groups. Wh
by tmoertel 2y ago
> And a final note, predicting one token at a time is what we are doing as humans when we speak...
I wouldn't be surprised if we could predict token groups. When speaking off the cuff, people often rely on well-worn phrases and cliches.
- ivalm 2y agoI think that corresponds to well worn phrases being a “single token”
- abakker 2y agoIndeed. An interesting reference to this is the work Millman Parry did to describe the key phrases in the Odyssey and the queues they gave to help someone memorize the poem. Also, this is maybe a semantic point, but, I am not predicting any words I speak. Not in a statistical sense. I have intent behind my words, which means I have an abstraction of meaning that I want to convey and I assemble the correct words to do that. no part of that is "predictive"