3 ms·
Question: why do people generate waveforms by neural networks? Would it be easier for a neural network to "control" an emulated throat/mouth and train against
by m1el 10y ago
Question: why do people generate waveforms by neural networks?
Would it be easier for a neural network to "control" an emulated throat/mouth and train against the sound output?
Emitting and post-processing waveforms entirely in neural networks seems like putting a lot of responsibilities into neural networks.
- svantana 10y agoThe reason is, the only speech data that is available in high quality and large quantities is in audio form. There has been research for decades on measuring or estimating the properties of the speech organ (oral cavity geometry etc), but that is largely an unsolved model inversion problem. Existing physical speech synthesis models are pretty crude and sound quite robotic.
- m1el 10y ago> Existing physical speech synthesis models are pretty crude and sound quite robotic. Could you provide some links/publications please?
- jhegedus42 10y agoI like that you mentioned "inversion". ML is strongly related to solving inverse problems but it does not tend to be emphasised enough, in my opinion. In reality, speech organs (a relatively low dimensional system) generate high dimensional data (speech). This is an example of the so called curse of dimensionality. What ML does is to invert this process and create a model of the generation process. My favorite example is quantum mechanics. Physicists made lot of measurements and created a model (quantum mechanics) that reproduces those measurements (and even works on test data, not only on training data). This is an example of a solution of an inverse problem. ML does the same. So in one way or an other, every working TTS has a model that accurately describes the speech organs and how the brain uses them to create speech from text.