7 ms·

While all of these vec2speech type models are impressive, I get the feeling that most of the comments didn't listen to any of the samples. It's still distinctly

While all of these vec2speech type models are impressive, I get the feeling that most of the comments didn't listen to any of the samples. It's still distinctly robotic sounding, probably has quite a bit of garbage output that needs to be filtered manually (as many of these nets often have) and is a far cry from fooling a human.

cel1ne 10y ago
It doesn't sound too different from a voice coming over a walkie talkie or some kind of intercom. The problem might be that high frequencies, especially overtones, aren't properly constructed, but I'm certain that can be improved.
- StavrosK 10y ago
  The main problem is that the algorithms don't yet know what to stress in a sentence. The problem is semantic, and not so much about the sound of the voice itself. You can synthesize someone's voice perfectly, but if it's stressing words incorrectly or not at all, it's not going to fool anyone. Then again, that's probably easier to work around by having humans annotate the sentences to be read.
  - comex 10y ago
    > Then again, that's probably easier to work around by having humans annotate the sentences to be read. Or by starting with a recording of someone else reading the sentence. Then you get the research problem known as "voice conversion", which has been studied a fair amount, but mostly prior to the deep learning era - and mostly without the constraint of limited access to the target person's voice. (On the other hand, research often goes after 'hard' conversions like male-to-female, whereas if your goal is forgery, you can probably find someone with a similar voice to record the input.) Anyway, here's an interesting thing from 2016, a contest to produce the best voice conversion algorithm, with 17 entrants: http://vc-challenge.org/summary.html http://vc-challenge.org/summary.html
  - truthexposer 10y ago
    pragmatic*, placing stress is less a problem of word meaning than it is of speaker adaptation for listener comprehension, emphasis, and prosodic tendencies. Even then, I don't believe the issue is with stress. I believe that the voices sound robotic because they are using, and also admitting because it makes their results impressive in some sense, very few samples, "less than a minute" they claim. Triphones are usually what speech systems are trained on. The amount of triphones (3-phoneme-grams) to cover a language's phonemic inventory is huge (50 phonemes = 50! triphones, which could mean a few hours of audio, although many will not occur within the language given the phonotactics of the language).
pluma 10y ago
Sure, it's not clean and crisp but if you ignore the distortions and the somewhat arbitrary intonation it's pretty close to the real thing. And I would argue that it's already good enough to fool grandma for a phone scam (but then again scammers currently get away with practically no resemblance at all).
- qeternity 10y ago
  Scammers have much more success phoning Granny and claiming that they are Windows Support or "The Internet" and simply asking for what they want. There are far easier ways to con than this, and for very sensitive scams where you need spear phishing of this order, then this will fall flat on its face. "Good enough" doesn't quite cut it when it comes to this. And fwiw, I think the intonations are actually impressively learned and not random. Trump's odd yet distinct intonations are capture quite well in their samples.
arghwhat 10y ago
Most of them are extremely mechanic, to the point where it's almost impossible to understand, but others are actually quite convincing. I think it primarily needs to learn to respect punctuation, and to translate them to a breathing pause that matches the target voice ("President having speech"-style long pauses vs. "Politician having their ass handed to them by journalist on TV"-no-air-needed pauses).
- got2surf 10y ago
  Absolutely - listening through the multiple samples with different intonation from both Obama and Trump, some of the samples are much more realistic, while others come off as robotic. Maybe it would be possible to train the system to prefer certain intonations in certain cases by rating the realism of the speech in context. It would be interesting to analyzes pauses around words grouped by word2vec! Or choosing a "style" of intonations based on punctuation, parameters like words/minute, etc.
noja 10y ago
Depends if you want to fool a person or a computer.
theprop 10y ago
Very impressive I velt -- though there were robotic artifacts here and there (well every several seconds), much of it was fairly natural and reasonably lo-fi quality convincing. There was a HN post a few weeks ago from Google of their most TTS algorithm which was the best I've heard I think, quite quite human (whose techniques were published and could be integrated into Lyrebird).
- squiggy22 10y ago
  you got a link anywhere?
  - nathanyukai 10y ago
    probably this one: https://deepmind.com/blog/wavenet-generative-model-raw-audio/ https://deepmind.com/blog/wavenet-generative-model-raw-audio...
    - veli_joza 10y ago
      I missed that one, and it's the best I heard so far. Bit of noise in the background, but the voice itself is very believable. I can imagine in year or two the radio and TV will start using this tech instead of actual speakers. Also, translation and synchronization of any content will be so much cheaper.
      - jpindar 10y ago
        Try listening to the NOAA weather radio. They've been using very good TTS for years.
        i336_ 10y ago
        Wow, they are. Very surprised. This is so good I'm wondering whether this is actually a massive (massive) sound bank. I think it might be a sound bank. A random radio receiver site I found that doesn't require Flash, tuned to NOAA for Akron OH: http://tunein.com/radio/NOAA-Weather-Radio-1624-s88289/ http://tunein.com/radio/NOAA-Weather-Radio-1624-s88289/ The list I got the above link from (^F "noaa"): http://tunein.com/radio/Weather-c100001531/ http://tunein.com/radio/Weather-c100001531/ I suspect the warbling I'm hearing is not due to TTS imperfections but 64kbps artifacting.
kastnerkyle 10y ago
I personally like this sample the most [0]. Note that these samples are not cherry-picked - having worked with very related algorithms [1] once it is trained well it pretty much "just works". There is a lot of room for DSP/hacks/tricks to improve audio quality - just the same as in concatenative systems, but the point of this demo is to show what is possible with raw data + deep learning. Also note that this is (as far as I am aware) learned directly on real data such as youtube, or recordings + transcripts. That is quite a bit different than approaches which require commercial grade TTS databases, which are generally professional speakers with more than 10 hours of speech each, and cost a lot of money. [0] https://soundcloud.com/user-535691776/special-guest-at-iclr https://soundcloud.com/user-535691776/special-guest-at-iclr [1] http://josesotelo.com/speechsynthesis/ http://josesotelo.com/speechsynthesis/
Infernal 10y ago
In the first clip, I'd say 80% of the soundbites were obviously robot-like, but one or two of the "Obama" quotes were startlingly clear - "The good news is, that they will offer the technology to anyone" - I can't hear anything wrong with that in the first clip at all. If they were all that quality I'd say we'd be easily fooled. As a proof of concept this is pretty big.
- bpicolo 10y ago
  I can definitely hear issues with that phrase. It has quite robotic drop-offs. Though coming soon: Neural networks to determine whether speech is NN-generated? :P
  - Macuyiko 10y ago
    > Neural networks to determine whether speech is NN-generated? I guess this would be an ideal use case for a generative adversarial network based approach.
  - blowski 10y ago
    In a sort of Turing Test where I don't know who's a robot, or where I'm not even expecting a robot, it would probably be a bit harder.
  - uremog 10y ago
    And then the generators train their own generation NN against that :P
    - hoschicz 10y ago
      And you have invented Generative Adversarial Networks. They are the basis of all new ML findings, like pix2pix.
  - robryk 10y ago
    This is likely part of how the speech-generating NNs are trained (ie. there's a generated-speech-detector and the network is trained to fool it, while it is also trained): https://arxiv.org/abs/1406.2661 https://arxiv.org/abs/1406.2661
qznc 10y ago
It might just be about the quality of the source. It seems they used public speeches. Obama sounds like he stands in a stadium. Trump is closer like a TV debate. Hillary sounds off to me, though.
- twothamendment 10y ago
  Hillary sounded very off, but what is new? Joking aside, Obama didn't sound too bad, but the Hillary voice was bad enough I wouldn't have included it in a demo.
ziikutv 10y ago
I think the editing manually is something people with resources and motives will do anyways. I can see something like this, if not already used, for propaganda. Tin Foil hat is on
nthcolumn 10y ago
I imagine that even at the point where it had been improved to such an extent that a human could not tell the difference it would still be possible to train a simple ai to tell which ones were non-human nevermind the original speaker.
anigbrowl 10y ago
I've been editing audio professionally for ~20 years now. This is short of perfect, but it's already good enough to be used in some kinds of editing emergencies - coincidentally exactly the ones where you're likely to have audio problems in the first place. For example, in a movie with a busy action scene involving gunfire/helicopters/storms the production audio is usually useless because there were fans and other loud machinery on the film set to create visual illusions of powerful winds and so on. In this situation the audio is either not recorded or used only as a guide track to match timing on a high-quality track recorded later in the studio. Actors hate re-recording dialog by lip-syncing in front of a screen and producers hate paying for it. This solution is already good enough to use for incidental characters in scenes that are going to be noisy anyway. I give it about 5 years before it's good enough to use in ordinary dialog scenes - not for the intimate conversation between the two Famous Actors flirting over a romantic dinner, but fine for replacing the waiter or other background characters.

cel1ne 10y ago

It doesn't sound too different from a voice coming over a walkie talkie or some kind of intercom. The problem might be that high frequencies, especially overtones, aren't properly constructed, but I'm certain that can be improved.

StavrosK 10y ago
The main problem is that the algorithms don't yet know what to stress in a sentence. The problem is semantic, and not so much about the sound of the voice itself. You can synthesize someone's voice perfectly, but if it's stressing words incorrectly or not at all, it's not going to fool anyone. Then again, that's probably easier to work around by having humans annotate the sentences to be read.
- comex 10y ago
  > Then again, that's probably easier to work around by having humans annotate the sentences to be read. Or by starting with a recording of someone else reading the sentence. Then you get the research problem known as "voice conversion", which has been studied a fair amount, but mostly prior to the deep learning era - and mostly without the constraint of limited access to the target person's voice. (On the other hand, research often goes after 'hard' conversions like male-to-female, whereas if your goal is forgery, you can probably find someone with a similar voice to record the input.) Anyway, here's an interesting thing from 2016, a contest to produce the best voice conversion algorithm, with 17 entrants: http://vc-challenge.org/summary.html http://vc-challenge.org/summary.html
- truthexposer 10y ago
  pragmatic*, placing stress is less a problem of word meaning than it is of speaker adaptation for listener comprehension, emphasis, and prosodic tendencies. Even then, I don't believe the issue is with stress. I believe that the voices sound robotic because they are using, and also admitting because it makes their results impressive in some sense, very few samples, "less than a minute" they claim. Triphones are usually what speech systems are trained on. The amount of triphones (3-phoneme-grams) to cover a language's phonemic inventory is huge (50 phonemes = 50! triphones, which could mean a few hours of audio, although many will not occur within the language given the phonotactics of the language).

pluma 10y ago

Sure, it's not clean and crisp but if you ignore the distortions and the somewhat arbitrary intonation it's pretty close to the real thing. And I would argue that it's already good enough to fool grandma for a phone scam (but then again scammers currently get away with practically no resemblance at all).

qeternity 10y ago
Scammers have much more success phoning Granny and claiming that they are Windows Support or "The Internet" and simply asking for what they want. There are far easier ways to con than this, and for very sensitive scams where you need spear phishing of this order, then this will fall flat on its face. "Good enough" doesn't quite cut it when it comes to this. And fwiw, I think the intonations are actually impressively learned and not random. Trump's odd yet distinct intonations are capture quite well in their samples.

arghwhat 10y ago

Most of them are extremely mechanic, to the point where it's almost impossible to understand, but others are actually quite convincing. I think it primarily needs to learn to respect punctuation, and to translate them to a breathing pause that matches the target voice ("President having speech"-style long pauses vs. "Politician having their ass handed to them by journalist on TV"-no-air-needed pauses).

got2surf 10y ago
Absolutely - listening through the multiple samples with different intonation from both Obama and Trump, some of the samples are much more realistic, while others come off as robotic. Maybe it would be possible to train the system to prefer certain intonations in certain cases by rating the realism of the speech in context. It would be interesting to analyzes pauses around words grouped by word2vec! Or choosing a "style" of intonations based on punctuation, parameters like words/minute, etc.

noja 10y ago

Depends if you want to fool a person or a computer.

theprop 10y ago

Very impressive I velt -- though there were robotic artifacts here and there (well every several seconds), much of it was fairly natural and reasonably lo-fi quality convincing. There was a HN post a few weeks ago from Google of their most TTS algorithm which was the best I've heard I think, quite quite human (whose techniques were published and could be integrated into Lyrebird).

squiggy22 10y ago
you got a link anywhere?
- nathanyukai 10y ago
  probably this one: https://deepmind.com/blog/wavenet-generative-model-raw-audio/ https://deepmind.com/blog/wavenet-generative-model-raw-audio...
  - veli_joza 10y ago
    I missed that one, and it's the best I heard so far. Bit of noise in the background, but the voice itself is very believable. I can imagine in year or two the radio and TV will start using this tech instead of actual speakers. Also, translation and synchronization of any content will be so much cheaper.
    - jpindar 10y ago
      Try listening to the NOAA weather radio. They've been using very good TTS for years.
      - i336_ 10y ago
        Wow, they are. Very surprised. This is so good I'm wondering whether this is actually a massive (massive) sound bank. I think it might be a sound bank. A random radio receiver site I found that doesn't require Flash, tuned to NOAA for Akron OH: http://tunein.com/radio/NOAA-Weather-Radio-1624-s88289/ http://tunein.com/radio/NOAA-Weather-Radio-1624-s88289/ The list I got the above link from (^F "noaa"): http://tunein.com/radio/Weather-c100001531/ http://tunein.com/radio/Weather-c100001531/ I suspect the warbling I'm hearing is not due to TTS imperfections but 64kbps artifacting.

kastnerkyle 10y ago

I personally like this sample the most [0]. Note that these samples are not cherry-picked - having worked with very related algorithms [1] once it is trained well it pretty much "just works". There is a lot of room for DSP/hacks/tricks to improve audio quality - just the same as in concatenative systems, but the point of this demo is to show what is possible with raw data + deep learning. Also note that this is (as far as I am aware) learned directly on real data such as youtube, or recordings + transcripts. That is quite a bit different than approaches which require commercial grade TTS databases, which are generally professional speakers with more than 10 hours of speech each, and cost a lot of money. [0] https://soundcloud.com/user-535691776/special-guest-at-iclr https://soundcloud.com/user-535691776/special-guest-at-iclr [1] http://josesotelo.com/speechsynthesis/ http://josesotelo.com/speechsynthesis/

Infernal 10y ago

In the first clip, I'd say 80% of the soundbites were obviously robot-like, but one or two of the "Obama" quotes were startlingly clear - "The good news is, that they will offer the technology to anyone" - I can't hear anything wrong with that in the first clip at all. If they were all that quality I'd say we'd be easily fooled. As a proof of concept this is pretty big.

bpicolo 10y ago
I can definitely hear issues with that phrase. It has quite robotic drop-offs. Though coming soon: Neural networks to determine whether speech is NN-generated? :P
- Macuyiko 10y ago
  > Neural networks to determine whether speech is NN-generated? I guess this would be an ideal use case for a generative adversarial network based approach.
- blowski 10y ago
  In a sort of Turing Test where I don't know who's a robot, or where I'm not even expecting a robot, it would probably be a bit harder.
- uremog 10y ago
  And then the generators train their own generation NN against that :P
  - hoschicz 10y ago
    And you have invented Generative Adversarial Networks. They are the basis of all new ML findings, like pix2pix.
- robryk 10y ago
  This is likely part of how the speech-generating NNs are trained (ie. there's a generated-speech-detector and the network is trained to fool it, while it is also trained): https://arxiv.org/abs/1406.2661 https://arxiv.org/abs/1406.2661

qznc 10y ago

It might just be about the quality of the source. It seems they used public speeches. Obama sounds like he stands in a stadium. Trump is closer like a TV debate. Hillary sounds off to me, though.

twothamendment 10y ago
Hillary sounded very off, but what is new? Joking aside, Obama didn't sound too bad, but the Hillary voice was bad enough I wouldn't have included it in a demo.

ziikutv 10y ago

I think the editing manually is something people with resources and motives will do anyways. I can see something like this, if not already used, for propaganda. Tin Foil hat is on

nthcolumn 10y ago

I imagine that even at the point where it had been improved to such an extent that a human could not tell the difference it would still be possible to train a simple ai to tell which ones were non-human nevermind the original speaker.

anigbrowl 10y ago

I've been editing audio professionally for ~20 years now. This is short of perfect, but it's already good enough to be used in some kinds of editing emergencies - coincidentally exactly the ones where you're likely to have audio problems in the first place. For example, in a movie with a busy action scene involving gunfire/helicopters/storms the production audio is usually useless because there were fans and other loud machinery on the film set to create visual illusions of powerful winds and so on. In this situation the audio is either not recorded or used only as a guide track to match timing on a high-quality track recorded later in the studio. Actors hate re-recording dialog by lip-syncing in front of a screen and producers hate paying for it. This solution is already good enough to use for incidental characters in scenes that are going to be noisy anyway. I give it about 5 years before it's good enough to use in ordinary dialog scenes - not for the intimate conversation between the two Famous Actors flirting over a romantic dinner, but fine for replacing the waiter or other background characters.