7 ms·
Bet it'd do just fine - maybe better than humans. People just get confused by large numbers of similar syllables because we have to buffer them for more proces
by marshray 4y ago
Bet it'd do just fine - maybe better than humans.
People just get confused by large numbers of similar syllables because we have to buffer them for more processing. I suspect a pure speech-to-text model doesn't need to worry so much about context and can just take the syllables one by one.
- fxtentacle 4y agoActually, this model uses attention layers which are kind of like a query/key=>value look-up and allow the model to merge the knowledge from neighboring time-steps into the current logit prediction. The result is that this model performs worse for highly repetitive words, just like humans do.
- marshray 4y agoThis stuff is so fascinating, I really need to learn more about it. :-)