4 ms·
One of the domains where realistic and fully customizable speech synthesis could usher in a renassaince of innovation, would be video games developement. I’ve
by Razengan 9y ago
One of the domains where realistic and fully customizable speech synthesis could usher in a renassaince of innovation, would be video games developement.
I’ve always been curious about how so seemingly little work has gone into speech generation, compared to graphics and physics and other technologies.
Problems with voice actors:
• The costs, often unaffordable by solo developers and small teams.
• All the text and the scripts have to be finalized and set in stone. You can’t change much, if at all, after you’ve finished recorded everything. Being able to re-hire the same actors for future sequels is also not guaranteed.
• Complete lack of real-time customization. For example, in MMORPGs and other online games. So we’re stuck with a few set phrases or using our own voice (not desirable all the time for everyone). Imagine if you could fully customize your voice as well as you can do with your character’s appearance, and have it realistically speak any text you type, or NPCs actually speaking your name instead of using generic placeholders.
• Players expect voice acting in all “major” titles. Plenty of games with great stories get skipped over if they don’t have voice acting.
- eduren 9y agoRelated: I've been looking for a good solution to voice synthesis with style transfer. One obstacle many potential Fallout 4 mods will face is that the protagonist is fully voiced. Imagine a mod author being able to create new voice lines in the same tone+style as the original voice actor. That does beg the question though; would a voice actor have legal grounds to stop such reproduction?
- sdenton4 9y agoOr better yet, just deploy the NN as a renderer for sounds, along with a bunch of text to render...
- ekianjo 9y ago> I’ve always been curious about how so seemingly little work has gone into speech generation, compared to graphics and physics and other technologies. The issue is coming first from the lack of assets. Voice samples that you can use for speech synthesis are not available easily, apart from your voice and a few friends' maybe. Then there's always been the issue that speech synthesis was not very good (we had working speech synthesis back in 1985 on the Amiga Workbench, but it was pretty much what you'd expect in terms of quality), and "not good enough" can break the experience. On top of that, speech synthesis is not an easy problem, it's definitely in the realm of machine learning, and video games development does not touch so much on machine learning so far. I think there's a ton of applications for machine learning in games (such as making less stupid AI, or more human-like AI), and speech synthesis is definitely one of them.
- Tobba_ 9y agoMachine learning really doesn't have many uses for games outside of the development cycle (or non-critical gimmicks). If it's exposed to player input - or especially; random input - it absolutely must have robust, stable and controllable behaviour, at least on paper. Most of the time getting any of those out of an NN is either difficult or blatantly impossible. Though speech synthesis specifically might be a good use case for some simpler ML, such as to form the final sound from processed phonemes (i.e acting as a fuzzy lookup table).
- Houshalter 9y agoIt may not be applicable in all games. But there are many games where the AI skill matters a lot to players. Or where they just get annoyed or bored with stupid AI. Games with bad AI (which is most games) frustrate me a great deal. AoE 2 let players make their own custom AIs. They took one of the best fan made AI scripts and made it the default AI in the new release. And players loved it. Because it provided a much harder and human like challenge, and didn't cheat like the old AI did. Obviously you shouldn't just train an NN to instant headshot players with optimal strategy. You can add realistic handicaps to the AI like noisy controls and human reaction times, so it isn't superhuman.
- Tobba_ 9y agoEnemy AI is most certainly the perfectly wrong place to apply ML due to the wide range of possible inputs (scenarios) and being in the utterly wrong complexity class (I mean, it's ostensibly doable with an RNN, but definitely not worth it). GOAP is still pretty much the end-all of game AI, but it can be pretty tricky to work with.
- XorNot 9y agoI feel like once there's something open source/free which works serviceabley for a big enough range of voices, you might see some mod projects jumping in to it. Certainly the idea of being able to put together new single player content fully voiced solo sounds appealing as hell to me.
- richardw 9y agoI can see trouble in the political arena when "recordings" are generated. Not for legal reasons but some voters might be easily duped.
- PeterisP 9y agohttps://grail.cs.washington.edu/projects/AudioToObama/ https://grail.cs.washington.edu/projects/AudioToObama/ is a nice example. The tech is already there, we can expect the psyops people of any major nationstate and/or any political campaign with significant financial resources to be able to generate such fake content in 2018.
- Razengan 9y agoCombined with realistic “Animoji”-like virtual face masks for digital doppelgängers, yes.
- emmelaich 9y agoYes, games. And the other thing the internet is for.
- juanmirocks 9y ago> Being able to re-hire the same actors for future sequels is also not guaranteed. Wow, it was not in my radar that AI could replace voice actors' jobs, but your comment has opened me to this idea. To give an example, in Spain (I'm Spanish), in the Simpsons series after several seasons the much beloved voice actor for Homer Simpson died. They had to replace it ofc with another one. Yet, I know that many people in my circle started stopping watching the Simpsons at that time because they couldn't stand the new voice. Regardless of whether the new voice was "worse" or "better", it was certainly different, and we humans get weirded with that. In light of this, I could see animated pictures with fully AI-rendered voices. Incredible.
- jldugger 9y agoI seem to recall a company doing this for NPR personalities. It makes a lot sense -- you have one person doing a boatload of speaking in front of a QA controlled mic, and you generally have their transcripts. Pretty straightforward dataset for training. Sadly, can't find the podcast that discussed this right now.
- nl 9y agoI assume you know about https://lyrebird.ai/ https://lyrebird.ai/ Lyrebird allows you to create a digital voice that sounds like you with only one minute of audio.
- juanmirocks 9y agoYes, just tried to record my voice, but the service didn't finish. Can be a temporal glitch. The fake voice by Donald Trump is terribly close. This definitely has a repercussion for fake news and overall trust. We will just not be able to trust what we see (computer-rendered images) or hear (rendered voice).