11 ms·
WaveNet launches in the Google Assistant
- samstave 9y agoBeuatiful, but for certain voices.. seems very close to how computerized it sounds.... except Japanese.... I'd like to be able to record my voice - and have it translated to text, then compare how it sounds via both engines vs my voice/cadence.
- CoolGuySteve 9y agoI'm interested in using similar generative adversarial networks to reduce artifacting in video streams. For example, highly compressed streams tend to show blocking artifacts on dark scenes, gradients, and static that could be smoothed in the decoder. I haven't actually done much about it yet, but I'm interested.
- nicklovescode 9y agoAdding matching of the laplacian to the optimization could be really helpful for this. I found this paper the other day and really enjoyed it https://arxiv.org/abs/1707.01253 https://arxiv.org/abs/1707.01253
- mrguyorama 9y agoTo be honest, it still obviously sounds machine generated. I guess it's a slight improvement, but the examples shown do not include any challenging words or phrases. We've been able to generate adequate sounding speech for simple phrases for quite some time. I bet this was really fun to work on however.
- mtgx 9y agoI thought it's just me. I remember the first WaveNet demo sounded significantly more natural than the non-WaveNet stuff. But now they sounded almost the same. The main difference is that with WaveNet you don't hear that robotic tone at the end of a phrase, that's typical of old TTS technologies. But the way the phrase is spoken still sounds like a machine said it, rather than a human. I wonder what compromises they made to improve the performance by 1,000x. There have to be some.
- sgk284 9y agoIf you ever listen to people try to record sound that's clear and precise, they actually sound fairly robotic. See this Google 20% Project where they explore Google Assistant's voice creation: https://youtu.be/qnGNfz7JiZ8?t=5m23s https://youtu.be/qnGNfz7JiZ8?t=5m23s WaveNet is probably modeling the source data very well. It sounds like they just need more data with emotion and inflection, rather than having source data that is optimized for monotonicity and precision.
- Yhippa 9y agoThe improved voices sound exactly like the audio from CBT's I've taken in the past. Could possibly have fooled me.
- AndrewKemendo 9y agoGreat video! If you work in radio or voiceover you learn really quickly that the voice is so much more complex than people give it credit for. Subtle changes in delivery, timing, inflection, syllabic emphasis, pauses etc... make a massive difference. Anyone can "talk like a robot" but speaking naturally is way more dynamical than just making the sounds of the words transition smoothly. I'm not sure if it's way easier or way harder than we're doing it now.
- ehsankia 9y agoI guess the hope is that even if it doesn't sound much better, this will open new doors, such as being able to iterate faster on new languages, new voices and new speaking styles (singing, whispering, etc). EDIT: A big example of that is the Japanese example at the bottom. English has had far more effort put into the old model, so it's already pretty good. But the difference between the old and new Japanese voice is really striking, and they were most likely able to make the new one much faster.
- Raphmedia 9y agoThe Japanese one is miles ahead of the previous one however.
- icc97 9y agoPerhaps I'm just less demanding but I was pretty blown away. Perhaps it sounds robotic but it's coming from the same voice recording. The main difference was how perfectly the words blend together. Now I'm just waiting to get Stephen Fry samples instead.
- supermdguy 9y agoWow. I can't believe they got such a significant speed increase.
- jay-anderson 9y agoHopefully there will be enough details on how they accomplish this when they release their paper.
- sanxiyn 9y agoIt has been known for some time that WaveNet can be greatly sped up, and exponentially so. See https://arxiv.org/abs/1611.09482 https://arxiv.org/abs/1611.09482 (Fast Wavenet) for an example.
- visarga 9y agoCan I get this on Mac OS as system voice?
- Tade0 9y agoI hope somebody uses this to immortalize Sir David Attenborough or the sadly departed Don LaFontaine.
- deleted 9y ago[deleted]
- spuz 9y agoOh man, I can already see the court cases of a certain robot's voice sounding a little too similar to a deceased human's that it should make royalty payments.
- Tade0 9y agoHeh, in my original post I wanted to follow up that paragraph with a question: Is someone's voice their IP? Would e.g. Don LaFontaine's family go all J.R.R. Tolkien and forbid the use of his voice in certain contexts? Then again, for now, it seems that it's easy to get away with synthesizing voices of popular characters as long as you don't use copyrighted names: https://acapela-box.com/AcaBox/index.php https://acapela-box.com/AcaBox/index.php (choose English(USA) - "Little Creature". Borked on Firefox but works in Chrome) "Little Creature" my ass. Apparently, names are IP, voices not so much. But we can't have nice things because like you said - eventually, someone's going to figure out how to file an effective lawsuit.
- icebraining 9y agoI don't think voices are copyrightable, but the recordings used to analyze them are, and so the resulting voice might be considered a derivative work.
- hughes 9y agoI don't see it as being very different than using someone's face to promote something. For example, should a lifelike digital model (or even a still image) of an actor's face still generate royalties for the actor's family after their death? Both cases are using a unique attribute of the person to promote something.
- taf2 9y agoIs there an api for this like amazon Polly?
- vagab0nd 9y agoI was wondering about that too. Google already has speech recognition APIs. I don't understand why they don't provide TTS.
- svantana 9y agoWhile the 100x speedup sounds impressive, a raw speed number without details regarding hardware is pretty meaningless. I'm guessing they got the 20x realtime speed from running it on their new TPU hardware, which they say can do 180 Tflops. That means you would need 9 Tflops of computing power to run this in realtime -- still pretty far away from running on a phone, or PC for that matter.
- chmod775 9y agoThey literally said it's launching in the Google Assistant (i.e. running on phones) now. In fact that's the title of the article.
- kowdermeister 9y agoHe also missed the speedup by a factor of 10, it's 1000x faster now :) I doubt it's just more hardware, they must have optimized the networks seriously.
- svantana 9y agoOf course they did, I was merely challenging the implied message that it is now 'fast enough', that the speed problem is solved.
- wmf 9y agoDoes TTS run on the phone or the cloud?
- trevorstrohman 9y agoIt runs in the cloud.
- chmod775 9y agoIt runs on the phone afaik, while speech recognition on the other hand is supported by stuff Google runs in the cloud. Edit: I might be wrong, at the end of a paragraph they say it runs on Google’s TPU cloud infrastructure, though it isn't clear to me whether they just use that for training. Edit 2: I just tried it on my phone. At least stuff like asking it to "Turn on WiFi" works without an internet connection, and yields a TTS response.
- mannigfaltig 9y agoI am wondering what their baseline is. They call it "Current Best Non-WaveNet". Quite frankly, Apple's most recent deep learning-based speech synthesis sounds superior, but there aren't enough samples to for a proper comparison: https://machinelearning.apple.com/2017/08/06/siri-voices.html https://machinelearning.apple.com/2017/08/06/siri-voices.htm...
- microcolonel 9y agoIt could just be a matter of opinion, but I prefer both Google's unit selection synthesis, and their WaveNet synthesis. The prosody in Apple's latest method is still annoying, nowhere near as good as the Google models of 2015 and 2016, and not remotely comparable to the WaveNet models. Apple's change in voice talent is an improvement though, and they may have more units than before, which is helpful. I believe their model also works offline, which is a huge plus (though I think Google's prior model works offline as well).
- CobrastanJorji 9y ago> prosody I learned a useful new word, thank you!
- deleted 9y ago[deleted]
- horsawlarway 9y agoI think the voice for the samples in your link still has the problems they talk about in that article. There are noticeable blips in the speech that sound unnatural, particularly when certain sound combinations are used. The very first sample with "Bruce Frederick" is clearly off. The intonation and timing between the end of bruce and the beginning of frederick is... mechanical. There's a similar problem in the OPs link with the non-wavenet English voice 1 when it says "Wavenet". Those issues are much less apparent in the wavenet voices. Timing problems are less noticeable, intonation problems are less noticeable. Frankly, the voices there sound VERY good, compared to anything I've heard. That said, I completely agree that there's not enough samples there to make any real judgement.
- DanAndersen 9y agoI'm most interested in its potential for audiobooks, especially of the vast array of old and less-common books that don't have human-made audiobook equivalents. I find myself constrained by the limits of audiobook choices, which tends toward best-sellers lists or pop-sci. Current attempts to use text-to-speech to generate audiobooks results in something that's frankly unlistenable. If TTS could get to a "good enough" point for audiobooks, that would open up a huge range of less-common content.
- michael_h 9y agoThis was kind of the target of the 2016 Blizzard Challenge (http://festvox.org/blizzard/blizzard2016.html http://festvox.org/blizzard/blizzard2016.html), as the training data was children's books (clean audio, 'reading voice' prosody, etc). Some of the voices that came out of that were incredible.
- exhilaration 9y agoOne of my co-workers told me in 2012 that he was doing exactly this. He used an ebook reader to download free ebooks from Gutenberg and then the IVONA text to speech engine for Android to listen to them on his drives. He had already finished a few classics like Treasure Island this way. I'm sure things have improved significantly since 2012 so what you're looking for is probably easily done.
- 9y ago
- denfromufa 9y agois there open-source implementation on github?
- erichocean 9y agoThere are two reachable from here: https://arxiv.org/abs/1611.09482 https://arxiv.org/abs/1611.09482
- baxuz 9y agoAwesome. I guess. Hard to know when everything AI/Assistant related is currently focused on the US market.
- tymekpavel 9y agoI've always wondered why companies don't just take all the close-captioned TV streams and use that as training data for their voice models. Seems like it would create a much more natural sounding voice model (at least as far as humans are accustomed to).
- epmaybe 9y agoWhat happens if the CC isn't entirely in sync with the video or audio?
- gwern 9y agoStrictly speaking, the text is never 'entirely in sync' because spoken words inherently blur together and are seamless; individual letters in the text do not start and end at precise intervals. This is one of the things that makes speech recognition so hard: letters, syllables, and words do not really exist as discrete things on the raw audio level. So this problem exists for any speech transcription dataset. To provide a loss function, then, you would use something like CTC: http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=FD2D37798AA9ABA95F00F67FA604F1AD?doi=10.1.1.75.6306&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=FD2... Fortunately, NNs are good at handling noisy data, and in practice they work very well for speech recognition/transcription.
- eginhard 9y agoRecordings are force-aligned to the transcriptions anyway (using essentially a speech recognition system) to obtain phone-level alignments. You don't need explicit timing information beforehand.
- SiVal 9y agoI think a better data source would be professionally produced audiobooks. Better, clearer voices, nearly perfect "transcription", enormous supply already recorded on a wide range of topics potentially available from one or two sources, etc.
- eginhard 9y agoEven if there is no background noise present, the quality is nowhere near that of professional studio recordings and would be very noticeable in the output. Also, for traditional systems you need a lot of data from one speaker only, they can't take advantage of other speakers' recordings (although WaveNet does that now). And "TV natural" might not be the style of natural you want from a TTS system.
- microcolonel 9y agoWow, I hear a huge improvement in the Japanese model: the difference between a robot in person and a young woman on the phone.
- ericlewis 9y agothis was what I noticed too, I don't really have an ear for Japanese, but it sounded as natural as when I have heard it spoken by others. Huge huge improvement.
- r00fus 9y agoThis is interesting - I don't speak much Japanese but that's the one where the difference in quality was most apparent IMHO.
- aikinai 9y agoIt's a huge improvement, but still nowhere near the state of the art for Japanese TTS which has always been ahead of English (since the phonetics are much simpler). Listen to the samples here for example: http://voicetext.jp/samplevoice/ http://voicetext.jp/samplevoice/
- microcolonel 9y agoMm, I see what you mean. It seems like voicetext.jp has more "correct" prosody, and seems to reproduce vocal fry correctly, I'm noticing that's missing from the WaveNet sample now. There's still some work to be done around filtering with voicetext, I hear pretty glaring artifacts between the units, whereas WaveNet doesn't produce any such artifacts.
- ryansama 9y agoThe Japanese wasn't so good. I like the sound of Risa in http://voicetext.jp http://voicetext.jp better. Try it out the Japanese Google used in their example. If someone knows of a better one it would be great to find out. 一つのウエーブネットのみで、多数の事なら、話者の音声波形極めて正確モデルができます。
- kuschku 9y agoSo, when can we expect an open source release of the tech and model?
- Sir_Cmpwn 9y agoNever, and it fucking sucks.
- imaginenore 9y agoDo you know any higher quality speech synthesis software?
- Sir_Cmpwn 9y agoNope.
- rjeli 9y agoDoes anyone have experience/success training WaveNet on their own?
- wmf 9y agoThis is Google, so they'll release a paper and then someone will clone it as an Apache project. Or maybe Linux Foundation these days.
- kuschku 9y agoI sure hope so, although I am extremely angered that they don’t release the model. Google is only having any success with this because they are getting training data from the public for free, they should also return the model to the public for free (at least for noncommercial use)
- wmf 9y agoAnd their Web ranking model and their spam filtering model? Face it, Google's gonna be Google. (And I don't think WaveNet is even based on public training data.)
- imaginenore 9y agoOh how much I wish Google paid to Morgan Freeman or David Attenborough to have their voice as an option.
- peteretep 9y agoWonder -- in all seriousness -- if the BBC will do Attenborough. If you're British and you've grown up with his documentaries, all other nature voices simply sound wrong.
- dharma1 9y agowow. I hope they will publish details on the optimisations
- alexasmyths 9y agoThese things are highly nuanced. The amount of tweaking and finessing that goes on under the hood for specific scenarios can change outcomes quite a lot. And from a product perspective, it can be taken quite far - for example, most of the 'common' things Siri says are not synthesized, it's literally are recording of the voice over artists. The more arcane stuff is synthesized. It's always comparing apples to oranges to bananas unless you really know what they're doing, even then it's hard.
- dzhiurgis 9y agoAny ideas when Assistant for iOS will released outside US?
- StavrosK 9y agoI would love something like this for computer notifications, or any sort of automated system notification. For a concrete example, my RC controller has the ability to do voice prompts (e.g. "landing gear down"), and it would be great if I could get a high-quality voice to speak all this. Hell, I'd settle for an API where I could send text and get high-quality voice back. Maybe I can somehow hack the Google Assistant app to do it...
- icebraining 9y agoIf the texts are fixed, why not just pay someone to record them? You can get 100 words for $5.
- StavrosK 9y agoMainly because that price is over the "can be bothered" threshold. I'm not going to find someone on fiverr and contract them to record a few words so I can have aural notifications for finished downloads, for example, but I might write a simple script to take the text and store an MP3 with the audio if it tacks a cent on my AWS bill.
- craigforster 9y agoI think AWS Polly (https://aws.amazon.com/blogs/aws/polly-text-to-speech-in-47-voices-and-24-languages/ https://aws.amazon.com/blogs/aws/polly-text-to-speech-in-47-...) can do this. I'm not sure of the quality compared to Google's or Apple's TTS though.
- komali2 9y agoI still don't understand the difference between Google Assistant and Google Now (or whatever), other than when I accidentally launch Assistant instead of Now, my commands are never understood.
- 2600 9y agoI don't understand the difference either other than Assistant seems to be replacing Now. The issue, I'm finding, is that Google Assistant requires that you turn on the recording of usage data and history to your account, such as Web & App activity, Location History, Voice & Audio activity. All recorded and tied to your account. I don't believe this was the case with Google Now. Google, to their credit, normally allows you to erase all this data and turn recording off, but with Google Assistant, they require it all to be on and recording. I avoid using it because of this restriction.
- obmelvin 9y agoConsider the information that Google Now was providing you before. It was almost certainly using this data.
- gvurrdon 9y agoIIRC one could use Google Now with voice recordings off and the extra-spying setting of "web and app activity" off (though web and app activity had to be on for many features). All this must now be on for the assistant, as you say. I did try out Google Now for a time due to having an Android Wear watch and found that as time went on more functionality required the turning on of these settings. One particular oddity for a time was that "OK Google, Navigate Home" would produce only a complaint about web and app activity being off, but "OK Google, navigate to $HOME_STREET, $HOME_TOWN" was fine. I've abandoned Android Wear because of this.
- SirZimzim 9y agoUi
- philtar 9y agoBut I'm imagining, but don't the wavenet versions sound very formal and a little depressed? As if it were a reflection of our society. Listen to them again. Very formal, lifeless and depressed.
- zebrafish 9y agoI think the logical next step for this is voice recognition.
- Abishek_Muthian 9y agoGuys, for those of you who would like to see how Microsoft Cognitive service’s TTS fares when compared to Google’s TTS. We had launched a Bot which gives voice summary of web content on Messenger, Slack, Telegram & Twitter with In-line audio player on first three. It’s great for sharing audio summary to our visually impaired friends. Check it out here - https://larynx.io/#larynxBot https://larynx.io/#larynxBot
- 0xbear 9y agoApple’s iOS 11 voice is pretty amazing too. I wonder what they are using.
- lawlessone 9y agoThis would be great for games with lots of speaking characters and other npcs. You could vary so much