4 ms·

Would this be a good target for deep learning? It's low level in some sense, but still nicely parametric. It strikes me that this could be a good synth for so

Would this be a good target for deep learning? It's low level in some sense, but still nicely parametric. It strikes me that this could be a good synth for some neural nets to learn how to play.

vortico 9y ago
Yes, speech recognition and speech generation would be easier to implement if you used neural networks that were trained on these vocal cord inputs rather than audio samples. In either case, you'd need to solve the inverse problem to generate vocal cord parameters given an audio sample. This seems difficult but I'd imagine some commercial software packages do it to some extent.
- smaddox 9y ago
  Or you could let the neural network solve the inverse problem for you, at least for speech generation.
- skykooler 9y ago
  I would imagine a neural network that fed parameter values to Voc would be far faster (real-time?) than something like WaveNet which needs to sample the output thousands of times a second.
  - zebproj 9y ago
    Correct. WaveNet is a very brute force approach to speech synthesis.
zebproj 9y ago
It seems like it would be, right? There are 44 tract-diameters you can modify to shape the vocal tract, and these can be used to generated specific vowel formants. I can imagine you can build a system using deep learning that can find the best parameters to match a steady state periodic pitch. It's a bit how some speech codecs work, like LPC10.
- woodson 9y ago
  Problem is, the output would sound pretty metallic, just like LPC10.
  - zebproj 9y ago
    Maybe, maybe not. LPC10 is a 8kHz speech codec optimized for low-bandwith signals. The Kelley-Lochbaum is a full-blown physical model of the tract. What you put into the filter is important. The LF glottal pulse model used here is a pretty good excitation signal... aspiration noise REALLY makes a difference. It would still sound artificial, but it definitely wouldn't sound metallic.

Would this be a good target for deep learning? It's low level in some sense, but still nicely parametric. It strikes me that this could be a good synth for some neural nets to learn how to play.

vortico 9y ago

Yes, speech recognition and speech generation would be easier to implement if you used neural networks that were trained on these vocal cord inputs rather than audio samples. In either case, you'd need to solve the inverse problem to generate vocal cord parameters given an audio sample. This seems difficult but I'd imagine some commercial software packages do it to some extent.

smaddox 9y ago
Or you could let the neural network solve the inverse problem for you, at least for speech generation.
skykooler 9y ago
I would imagine a neural network that fed parameter values to Voc would be far faster (real-time?) than something like WaveNet which needs to sample the output thousands of times a second.
- zebproj 9y ago
  Correct. WaveNet is a very brute force approach to speech synthesis.

zebproj 9y ago

It seems like it would be, right? There are 44 tract-diameters you can modify to shape the vocal tract, and these can be used to generated specific vowel formants. I can imagine you can build a system using deep learning that can find the best parameters to match a steady state periodic pitch. It's a bit how some speech codecs work, like LPC10.

woodson 9y ago
Problem is, the output would sound pretty metallic, just like LPC10.
- zebproj 9y ago
  Maybe, maybe not. LPC10 is a 8kHz speech codec optimized for low-bandwith signals. The Kelley-Lochbaum is a full-blown physical model of the tract. What you put into the filter is important. The LF glottal pulse model used here is a pretty good excitation signal... aspiration noise REALLY makes a difference. It would still sound artificial, but it definitely wouldn't sound metallic.