4 ms·

I came here to say this. My brother has a PhD in chemistry and no coding experience. He was able to create a voice model of himself using basic nvidia example g

I came here to say this. My brother has a PhD in chemistry and no coding experience. He was able to create a voice model of himself using basic nvidia example generators in a week. My dad lost his voice and it would have been very nice to have a TTS that was much more close to him. I personally would think it would be worth it to have that database. But obviously also attend to the human matters as well, eg spend time.

tombert 6y ago
This sounds pretty cool (your brother making the voice model, not your dad losing the voice)...do you have a link to this example? I would love to play with this.
audiohermit 6y ago
I work in pathological speech processing/synthesis so I'm unfortunately familiar with your father's position. It really sucks that these people didn't know that archiving their voice would've been useful. I hear snippets that people manage to glean from family videos right after listening to their current voices and it makes me really sad. On the upside, your father can choose any celebrity he wants to voice him! Tons of celeb data is publicly available (VoxCeleb 1 & 2).
- vervez 6y ago
  Is Morgan Freeman the most used celebrity?
  - deleted 6y ago
    [deleted]
  - Deinos 6y ago
    A ranking of used voices would be fascinating. Especially broken down by user statistics.
  - core-questions 6y ago
    I'd go for Stephen Hawking, myself. (Not using his voice synth, reconstructed using ML, because it should sound more natural that way ;-)
    - shagie 6y ago
      I recall that the "say" program on the SGI from the mid 90's was approximately Hawking's voice. Hawking gave his speech for the Whitehouse Millennium Lecture at SGI also, and while I wasn't able to attend I found the transcript of it and fed it in there... there were some jokes that he had that only really came through with the intonation and pacing of a voice synth -- its the ultimate dead pan voice. https://clinton.presidentiallibraries.us/items/show/16112 https://clinton.presidentiallibraries.us/items/show/16112 https://youtu.be/orPUQm1ZRSI https://youtu.be/orPUQm1ZRSI And his voice was his - even with the American accent. https://www.news.com.au/technology/innovation/why-stephen-hawkings-voice-computer-spoke-with-an-american-accent/news-story/d4529ffb6341278d8c1b33e06cd3099c https://www.news.com.au/technology/innovation/why-stephen-ha... > “It is the best I have heard, although it gives me an accent that has been described variously as Scandinavian, American or Scottish.” > ... > “It has become my trademark and I wouldn’t change it for a more natural voice with a British accent. > “I am told that children who need a computer voice want one like mine.” Somewhere, I recall a NOVA(?) program from the mid 80s where it showed him using the speech synthesizer and the thing that he said with it that still sticks in my mind is the "please excuse my American accent". In later years he was given the opportunity to upgrade it to a more natural sounding voice - but that voice was his.
      - hapless 6y ago
        Hawking’s original voice synth was the default sound of the DECtalk hardware speech synth. It would not surprise me if SGI’s software implementation were similar to the he most popular hardware of the 1980s.
      - egypturnash 6y ago
        Near the end of his life, his original voice computer started to fall apart. He managed to get in touch with the people who wrote the software, who started a mad scramble to find source, and ultimately ended up emulating the whole setup on a Pi. https://theweek.com/articles/769768/saving-stephen-hawkings-voice https://theweek.com/articles/769768/saving-stephen-hawkings-...
- clan 6y ago
  Are there any simple howtos anywhere which describes the process in as simple terms as possible? Without knowing the cool toolkits du jour. Something like: - Download these texts - Record in WAV at least 48 kHz - Record each line in a separate file. - Do 3 takes of each line: flat, happy, despair Maybe even a minimal set and a full set depending on how much effort you are willing to put in. A plain description on how to capture a raw base which within reason and technology could be used as a baseline for the most common toolkits. I have myself looked into this (for fun) but I felt I needed a very good understanding of the toolkits before even starting to feed in data. And for my admittedly unimportant use it seemed a huge investment to create a corpus I was not even confident would work. I ended up taking the low road and used an existing voice.
  - audiohermit 6y ago
    Not really, this is the only thing I know of in terms of collection: https://www.isca-speech.org/archive/Interspeech_2018/pdfs/2400.pdf https://www.isca-speech.org/archive/Interspeech_2018/pdfs/24... Usually you're basing your recipe off of those for existing datasets (TIMIT, WSJ, LibriSpeech, etc).
  - voicevoice50 6y ago
    For recording training audio: https://github.com/daanzu/speech-training-recorder https://github.com/daanzu/speech-training-recorder The recorder works with Python 3.6.10. Need to pip install webrtcvad also.
- grogenaut 6y ago
  Unfortunately dad passed 21 years ago. But the options now are much better. Just projecting my past experiences on the obvious Delta.
dheera 6y ago
Which generator works the best, qualitatively? I come from a vision/ML background but haven't played with speech at all, so it's completely new to me, and wondering what the state of the art is. I've been wanting to create a TTS of myself so I can take phone calls using headphones and type back what I want to say so that I don't have to yell private information out loud in public locations. Would be nice if during non-COVID times I could sit in a train seat and take phone calls completely silently.
- audiohermit 6y ago
  Much of the work in speech synthesis has been about closing the gap in vocoders, which take a generated spectrogram and output a waveform. There's a clear gap between practical online implementations and computational behemoths like WaveNet. As you implied it's hard to quantitatively judge which result is better, papers usually use surveys to judge. Here's a recent work that has a good comparison of some vocoders: https://wavenode-example.github.io/ https://wavenode-example.github.io/ Edit: WaveRNN struck a good balance for me in the past but is not shown in the link. Tons of new work coming out though!
  - sdenton4 6y ago
    WaveRNN (and even slimmer versions, like LPCNet) are great, and run for a tiny fraction of the compute of the original WaveNet. Pruning is also a good way to reduce model sizes. I'm not sure what's up with the WaveGLOW (17.1M) example in the linked wavenode comparison... The base WaveGLOW sounds reasonable, though. They're also using all female voices, which strikes me as dodgy; lower male voice pitch tracking is often harder to get right, and a bunch of comparisons without getting into harder cases or failure modes makes it seem like they're covering something up. (I've run into a bunch of comparisons for papers in the past where they clearly just did a bad job of implementing the prior art. There should be a special circle of hell...)
    - audiohermit 6y ago
      Agreed. I didn't have a better comparison at hand. I'm looking at you GAN papers.

tombert 6y ago

This sounds pretty cool (your brother making the voice model, not your dad losing the voice)...do you have a link to this example? I would love to play with this.

audiohermit 6y ago

I work in pathological speech processing/synthesis so I'm unfortunately familiar with your father's position. It really sucks that these people didn't know that archiving their voice would've been useful. I hear snippets that people manage to glean from family videos right after listening to their current voices and it makes me really sad. On the upside, your father can choose any celebrity he wants to voice him! Tons of celeb data is publicly available (VoxCeleb 1 & 2).

vervez 6y ago
Is Morgan Freeman the most used celebrity?
- deleted 6y ago
  [deleted]
- Deinos 6y ago
  A ranking of used voices would be fascinating. Especially broken down by user statistics.
- core-questions 6y ago
  I'd go for Stephen Hawking, myself. (Not using his voice synth, reconstructed using ML, because it should sound more natural that way ;-)
  - shagie 6y ago
    I recall that the "say" program on the SGI from the mid 90's was approximately Hawking's voice. Hawking gave his speech for the Whitehouse Millennium Lecture at SGI also, and while I wasn't able to attend I found the transcript of it and fed it in there... there were some jokes that he had that only really came through with the intonation and pacing of a voice synth -- its the ultimate dead pan voice. https://clinton.presidentiallibraries.us/items/show/16112 https://clinton.presidentiallibraries.us/items/show/16112 https://youtu.be/orPUQm1ZRSI https://youtu.be/orPUQm1ZRSI And his voice was his - even with the American accent. https://www.news.com.au/technology/innovation/why-stephen-hawkings-voice-computer-spoke-with-an-american-accent/news-story/d4529ffb6341278d8c1b33e06cd3099c https://www.news.com.au/technology/innovation/why-stephen-ha... > “It is the best I have heard, although it gives me an accent that has been described variously as Scandinavian, American or Scottish.” > ... > “It has become my trademark and I wouldn’t change it for a more natural voice with a British accent. > “I am told that children who need a computer voice want one like mine.” Somewhere, I recall a NOVA(?) program from the mid 80s where it showed him using the speech synthesizer and the thing that he said with it that still sticks in my mind is the "please excuse my American accent". In later years he was given the opportunity to upgrade it to a more natural sounding voice - but that voice was his.
    - hapless 6y ago
      Hawking’s original voice synth was the default sound of the DECtalk hardware speech synth. It would not surprise me if SGI’s software implementation were similar to the he most popular hardware of the 1980s.
    - egypturnash 6y ago
      Near the end of his life, his original voice computer started to fall apart. He managed to get in touch with the people who wrote the software, who started a mad scramble to find source, and ultimately ended up emulating the whole setup on a Pi. https://theweek.com/articles/769768/saving-stephen-hawkings-voice https://theweek.com/articles/769768/saving-stephen-hawkings-...
clan 6y ago
Are there any simple howtos anywhere which describes the process in as simple terms as possible? Without knowing the cool toolkits du jour. Something like: - Download these texts - Record in WAV at least 48 kHz - Record each line in a separate file. - Do 3 takes of each line: flat, happy, despair Maybe even a minimal set and a full set depending on how much effort you are willing to put in. A plain description on how to capture a raw base which within reason and technology could be used as a baseline for the most common toolkits. I have myself looked into this (for fun) but I felt I needed a very good understanding of the toolkits before even starting to feed in data. And for my admittedly unimportant use it seemed a huge investment to create a corpus I was not even confident would work. I ended up taking the low road and used an existing voice.
- audiohermit 6y ago
  Not really, this is the only thing I know of in terms of collection: https://www.isca-speech.org/archive/Interspeech_2018/pdfs/2400.pdf https://www.isca-speech.org/archive/Interspeech_2018/pdfs/24... Usually you're basing your recipe off of those for existing datasets (TIMIT, WSJ, LibriSpeech, etc).
- voicevoice50 6y ago
  For recording training audio: https://github.com/daanzu/speech-training-recorder https://github.com/daanzu/speech-training-recorder The recorder works with Python 3.6.10. Need to pip install webrtcvad also.
grogenaut 6y ago
Unfortunately dad passed 21 years ago. But the options now are much better. Just projecting my past experiences on the obvious Delta.

dheera 6y ago

Which generator works the best, qualitatively? I come from a vision/ML background but haven't played with speech at all, so it's completely new to me, and wondering what the state of the art is. I've been wanting to create a TTS of myself so I can take phone calls using headphones and type back what I want to say so that I don't have to yell private information out loud in public locations. Would be nice if during non-COVID times I could sit in a train seat and take phone calls completely silently.

audiohermit 6y ago
Much of the work in speech synthesis has been about closing the gap in vocoders, which take a generated spectrogram and output a waveform. There's a clear gap between practical online implementations and computational behemoths like WaveNet. As you implied it's hard to quantitatively judge which result is better, papers usually use surveys to judge. Here's a recent work that has a good comparison of some vocoders: https://wavenode-example.github.io/ https://wavenode-example.github.io/ Edit: WaveRNN struck a good balance for me in the past but is not shown in the link. Tons of new work coming out though!
- sdenton4 6y ago
  WaveRNN (and even slimmer versions, like LPCNet) are great, and run for a tiny fraction of the compute of the original WaveNet. Pruning is also a good way to reduce model sizes. I'm not sure what's up with the WaveGLOW (17.1M) example in the linked wavenode comparison... The base WaveGLOW sounds reasonable, though. They're also using all female voices, which strikes me as dodgy; lower male voice pitch tracking is often harder to get right, and a bunch of comparisons without getting into harder cases or failure modes makes it seem like they're covering something up. (I've run into a bunch of comparisons for papers in the past where they clearly just did a bad job of implementing the prior art. There should be a special circle of hell...)
  - audiohermit 6y ago
    Agreed. I didn't have a better comparison at hand. I'm looking at you GAN papers.