4 ms·

Anecdotally, I realized I was doing something like this when I was having trouble understanding people speaking in a noisy room, in a language I’m not so profic

Anecdotally, I realized I was doing something like this when I was having trouble understanding people speaking in a noisy room, in a language I’m not so proficient in. As you listen to someone, your brain is constantly matching the sounds arriving at your ears with a prediction of what the next few words might be. Listening in a non-native language, my predictions about what comes next aren’t very well tuned at all, so if I can’t hear every word clearly then I can easily get lost. Another signpost: sometimes you mishear someone — “oh, I thought you said xyz” — but the thing you thought you heard them say is never gibberish, it’s a grammatically and contextually valid way to complete the sentence.

kensai 4y ago
Very good point. This also applies for lip reading, especially important in languages one is not proficient in or has some hearing hindrance. It was especially hard in the COVID pandemic where people where wearing masks all the time.
MagicMoonlight 4y ago
There’s definitely a similarity with us in that you need to have been trained on enough data to build up that prediction. Language models are just missing some component that we have. The method for deciding what to output is wrong. People aren’t just guessing the next sound. It’s like they said, there’s multiple levels of thought and prediction going on. It needs some sort of scratch pad where it keeps track of states/goals. “I’m writing a book” “I want to make this character scary” Currently it only works on the next tokens and its context is the entire text so far, but that’s not accurate. I’m not deciding what to say based exactly on the entire text so far, I’m feature extracting and then using those features as context. e.g She looks sad but she’s saying she is fine and it’s to do with death because my memory says her dad died recently so the key features to use for generation are: her being sad, her dad died, she may not want to talk about it

kensai 4y ago

Very good point. This also applies for lip reading, especially important in languages one is not proficient in or has some hearing hindrance. It was especially hard in the COVID pandemic where people where wearing masks all the time.

MagicMoonlight 4y ago

There’s definitely a similarity with us in that you need to have been trained on enough data to build up that prediction. Language models are just missing some component that we have. The method for deciding what to output is wrong. People aren’t just guessing the next sound. It’s like they said, there’s multiple levels of thought and prediction going on. It needs some sort of scratch pad where it keeps track of states/goals. “I’m writing a book” “I want to make this character scary” Currently it only works on the next tokens and its context is the entire text so far, but that’s not accurate. I’m not deciding what to say based exactly on the entire text so far, I’m feature extracting and then using those features as context. e.g She looks sad but she’s saying she is fine and it’s to do with death because my memory says her dad died recently so the key features to use for generation are: her being sad, her dad died, she may not want to talk about it