3 ms·
Wouldn't the simpler explanation be that the pretraining teaches the model to extract human voices and phonemes from general audio, and so extending the model t
by fenomas 4y ago
Wouldn't the simpler explanation be that the pretraining teaches the model to extract human voices and phonemes from general audio, and so extending the model to other languages is then easier because the model is already working from phonemes?
- e12e 4y agoThat's probably what op meant by "it's like Chomsky" :)
- wongarsu 4y agoIt probably goes even beyond that. Most languages are related to other languages, and somebody good in German and Norwegian could do an ok job transcribing Swedish with minimal training. Similarly, learning English is a lot easier for German speakers than for Chinese, even though German and English are quite different in pronunciation. The more languages from a language family a human speaks, the easier it is for them to learn each additional language from that family. It stands to reason that deep learning can benefit similarly.