9 ms·
OpenVoice: Instant Voice Cloning
- alokjnv10 3y ago[flagged]
- chrra 3y agoThere's install instruction under “How to Use” and example code in the .ipynb files. It's mostly geared towards TTS, but it seems to work with voice recordings too.
- alokjnv10 3y ago[flagged]
- andy_ppp 3y agoI really can’t wait until voice cloning means we get a version of audiobooks read in the author’s voice. Of course it will never be quite as good as them reading it themselves but I think the author’s voice adds something that voice actors can’t- they appear to be too generic and too affected in their pronunciation for me to connect with.
- block_dagger 3y agoNever be as good as human? I disagree, seems like it’ll be nailed, no way to tell from the outside.
- smeej 3y agoWhat the author adds, if they're not also a trained/well-practiced voice author, is that their inflection exactly matches how they meant the words in the book to be spoken/understood. AI isn't going to be able to do that. As good as it may get, it won't be able to read the mind of the author. It's going to be even more generic than a human reader.
- throwthrowuknow 3y agoExactly, the improvement will be in rerecording terrible readings into something enjoyable or at least inoffensive. That and personalization so you can choose the voice that you prefer.
- wddkcs 3y agoWill this be the case when/if books become largely written with the help of AI? Let alone when AI start writing the books themselves.
- smeej 3y agoIt's an interesting question, but I don't think the models are trained to be spoken-language first like humans are. I think we all ("we" meaning "people who communicate by speaking," because I'm less sure if this would be true of sign languages) inherently think of writing as a somewhat lossy graphical representation of speech (which is itself an extremely lossy form of telepathy). But the LLMs don't, do they? If anything, they're written-first, or even on some deep level, binary-first. I don't think what they write even has "a way they meant the words to be emphasized in speech."
- wddkcs 3y agoSeems like a lot of guessing. I'm not convinced AI aren't "thinking" in the same manner we are. Eventually we'll have models trained on speech only, or modes of expression we can't fathom. Humans have no moat.
- jilijeanlouis 3y ago+1000
- smeej 3y agoI wonder if AIs would even decide to do things in the same way we do. Most of what humans do has come from generations of having to operate within constraints that change over time. AI gets to leapfrog those constraints for a whole different kind. Why would we assume what comes from them will even aspire to "being like humans"? The number of reasons AIs might not add the same things to an audio version of a book (the context we're talking about in this thread) is essentially infinite. It seems vastly more likely that they won't add what the author adds than that they will. Humanity may not have a moat, but each individual human does, especially when it comes to art, where I'd include writing.
- PodgieTar 3y agoOdd, because I actually worry about this. I don't see why you'd want your books read by the author. Trained Voice Actors do a much better job, and can modulate their voices based on tone. Autobiographies? Fine, but most of the time they are usually read by their authors.
- Rodeoclash 3y agoIf you think that a voice actor reading an audio book is too generic then I've got bad news about an AI trained on the author's voice...
- andy_ppp 3y agoI was hoping it would be voice transfer so the voice actor would give all the intonation and emotion and the AI would take that and make it sound like the author. Reading text with AI is getting better but yes it’ll be worse for a long time.
- joshstrange 3y agoI have nearly no desire to have my book read my the author. They are good at writing, and an audiobook is not simply “reading” the words on the page. Maybe something like Descript that the author can use to tweak pronunciation after it’s narrated but I don’t want the author’s voice. I would like train a model on Allyson Johnson’s voice (narrated the Honor Harrington books) and then use that to re-narrate the 1-2 books in one of the spinoffs (I think it was in the Saganami Island series?) where they used a different narrator (who was horrible). I also might be interested in using it to clean up the Wheel of Time series where, while it’s the same 2 narrators, they change the pronunciation of various names/words book-to-book. “Moghedien” being the one that stands out most. They pronounce it at least 3 different ways: * Mo-gid-e-on * Mo-ga-dean * Mog-a-din
- kornork 3y agoIt's curious, to me at least, why they didn't just go back and fix those themselves later. The early ones were on CD (or tape?), so maybe that's why.
- joshstrange 3y agoI also wonder that. I'm not an audiobook narrator but if I were I'd need a audio "library" of names/places/etc that I could refer back to before reading a passage with a word I can't remember how to pronounce. The source of that "library" could either be from the author and/or my previous pronunciations. Without that I'd have no idea how I would stay consistent.
- kornork 3y agoI think I'd prefer to have options for each audiobook. I have favorite narrators, and find others unlistenable. There are also thousands and thousands of books that will never otherwise be turned into audio format unless an AI is used.
- infoseek12 3y agoWriting and being a voice actor are two quite different skills. My experience with author narrated audiobooks is that there isn’t very much overlap.
- andrewstuart 3y agoAudiobooks in the authors voice….. fine for non fiction, usually terrible for fiction.
- nolok 3y agoWe're really in an era were laws and their enforcement will have a lot of catching up to do very fast. Fake historical proof, fake leaks, fake endorsments, fake ads ... People couldn't be bothered to double check when it was mere random text article on facetok, it's going to be so much worse ...
- throwthrowuknow 3y agoA digital audio file is not even close to being proof of anything. Even without voice cloning you can easily edit, clip and compose audio into almost anything you want. It’s also not difficult to simply impersonate someone else’s manner of speaking with practice something that is commonly done by both amateurs and professional actors. The only thing that changes is the ease with which this can be done which should help everyone understand how unreliable such “proof” is.
- colecut 3y agoWhen dealing with social conflicts, there is very rarely "proof", just evidence In courts of law, digital tapes have been frequently used as evidence.
- exe34 3y agoThe thing about evidence is that it can be contested. I imagine that in the cases you refer to, one side presented them as evidence and the other side said oh bugger, you got me. If they had reason to believe the audio had been tempered with, or entirely fabricated, they would then bring expert witnesses and eventually the other side would have to offer more evidence that it was not tampered - often this sort of thing comes down to "under oath, I swear I recorded this at the date/time and the subject on the tape is $name". This might not be enough to convict or exonerate, but it'll then count as part of the case.
- close04 3y ago> The thing about evidence is that it can be contested. The danger is in the court of public opinion where just having the conversation is damage enough. If you have to go out there and fight it then you give it even more visibility and it takes your focus and resources away from more useful activities. At the end of the day many people only remember the accusation, not what comes after.
- mindcrime 3y agoMy voice is my passport, verify me
- perakojotgenije 3y agonice reference :-)
- _joel 3y agoI loved that screensaver, LLAP.
- joshstrange 3y agoI always have to say that in my head when I hear the word “Passport”. Also I love this scene of social engineering: https://youtu.be/WdcIqFOc2UE?si=tP2HxVEskl9szuKO https://youtu.be/WdcIqFOc2UE?si=tP2HxVEskl9szuKO
- mindcrime 3y agoClassic. The whole "I'm late for the party on the second floor" scene is great as well.
- glandium 3y agoThe last example where the generated voice talks in different languages? They don't sound like the same voice at all. I know I don't sound the same when I speak French or Japanese, but the difference in the example is bigger.
- thimkerbell 3y agoHow does a society function, with lookalike tools like this around? How would we prepare for it?
- courseofaction 3y agoEstablish and constantly vett networks of trust? Only the source can be trusted now.
- squigz 3y agoThe problem is it's nearly too late to "prepare" for it - the time for that was years ago Instead, we've spent that time doing things that are only going to exacerbate the issues - like doubling down on voice as an ID More importantly, trust in both public and private institutions and even between citizens has slowly eroded - understandably so, if you ask me, but still - which as another commenter has pointed out is going to be even more important going forward. I imagine rebuilding that trust is going to be even more painful with stuff like this. (The only other alternative I can think of is along the lines of cryptographically-signed content from end-to-end, government ID to access the Internet, etc. This would be highly detrimental to a free society, so I hope we don't go this route)
- joshstrange 3y agoThe movie Sneakers sure would get a lot less interesting with voice cloning. And I’d miss out of one of my favorite/most funny scenes in the movie “I would really like to hear you say the word ‘Passport’” [0]. I cannot hear the word “Passport” without thinking of that scene and then I always have to say “My voice is my passport, verify me” in my head to complete the circle. [0] https://youtu.be/WdcIqFOc2UE?si=tP2HxVEskl9szuKO https://youtu.be/WdcIqFOc2UE?si=tP2HxVEskl9szuKO
- throwthrowuknow 3y agoDon’t take any wooden nickels
- deleted 3y ago[deleted]
- youngNed 3y ago[flagged]
- _joel 3y agoIt's not cloning, it's just copying the tonality. It states this in the docs but still calls itself voice cloning. I tried it and I ended up sounding American, not my ususal dulcet Lancashire tones. Absolutely nothing like me.
- youngNed 3y agonever really thought about how much i need a 'Fred Dibnah' voiced AI until now
- _joel 3y agoAye lad!
- unraveller 3y agoYou should be able to bring it back to your proper accent using https://voiceshopai.github.io https://voiceshopai.github.io VoiceShopAi can convert from young to old, male or female, or into any country's accent. found via https://github.com/metame-ai/awesome-audio-plaza https://github.com/metame-ai/awesome-audio-plaza who is tracking most things in the voice space as they come up.
- _joel 3y agoCan't see any code for that, unless it's there and I've missed it.
- unraveller 3y agoYou're right. No code ever. Demo looks to just be a tease for the commercial offering to come. It seems they think it is more ethical for people to live their lives misinformed by the availability of this tech because only good people will study the field or run the voice APIs. >we share the details of our findings here, but do not plan to publicly release the model checkpoints or implementation at this time [due to deepfake potential]
- 3y ago
- peter_d_sherman 3y agoRelated: https://github.com/topics/voice-clone https://github.com/topics/voice-clone
- exe34 3y agoDo you know of any of these that actually works? So far every time I've tried one, it just sounded like a random new voice neither the target (me) nor the source(literally any voice).
- spmurrayzzz 3y agoElevenLabs[1] is actually very good, and may have gotten better since the last time I tried it, but its neither free nor is it open source/weights. [1] https://elevenlabs.io/ https://elevenlabs.io/
- Draiken 3y agoMaybe I'm doing something wrong, but I "cloned" my own voice and the outputs were nothing like my voice. I thought I was going to hear myself speaking French, but it's definitely not that. A bit misleading if none of the examples actually clone a voice when the title is "instant voice cloning"
- g4zj 3y agoI would love something like this for musical instruments. To be able to dial in the tone of my favorite artists using AI would be incredible.
- SoftTalker 3y agoProtools or another DAW will do that easily.
- vouaobrasil 3y agoWe've got a pretty sick society. We've spent so much money developing this technology to clone voices when we've got 8 billion voices on the planet today. We've developed 4K and even 8K video when we've already got beautiful scenes outside -- oh wait, we ruined those already with technology and roads and cars. We've got the smell of beautiful flowers and yet we have an entire chemical industry trying to replicate more artificial smells. We're sick and AI is the apex of that pathology. Can you imagine how nice life would be if we just devoted all this energy into providing for basic needs and focusing on social problems instead of technological ones? We're making the world worse with this shit.
- echelon 3y agoYour way of thinking is "sick", some kind glass-is-empty malaise, and you need to find a cure [1]. If you think art is perfect the way it is, you're not dreaming. I want a thousand science fiction epics. The world only has Star Wars crap and bad Dune adaptations. I want to take what's in my mind and put it to audio-visual, and play it for myself and friends. I don't want the tech and art of today. I was born too early. I want what's next. I have so many dreams, and for just once, I'd like them to start coming true. This tech is a first glimpse over that hill. [1] (Sorry, I'm not attacking you. I just wanted to use your language to grab attention. I believe this is too negative an outlook for something that will have incredible use cases.)
- relaxing 3y agoThere’s so much more SF out there, even in the space opera subgenre, than Star Wars and Dune. If you don’t know how to navigate that, there’s no way you’ll produce anything of worth. Start reading.
- Der_Einzige 3y agoThe original dune movie was excellent and I will die on that hill. It accurately portrays the absurdity of the books. You’d give the same complaints to anyone who bothered to try to film Charlie and the great glass elevator.
- teeray 3y ago“Introducing VoicePrint—our new way to secure your account! Speak with a representative to enable this feature today!”
- andsoitis 3y agoHow is it cloning when it doesn't preserve accent? Seems like overselling.
- ChildOfChaos 3y agoWhere are the best places to keep up with all of this? I'm very interested in this area as I want to use these tools to create things with and my own voice isn't great for this. Speech to speech seems like it might be better than TTS to get it to be more natural, i've played around with some tools like RVC etc, but I feel like there are maybe a lot of great AI workflows I am missing amoungst all the AI noise, it's the interesting workflows and people doing interesting things with AI that I am more interested in.
- jilijeanlouis 3y agoDefinitely twitter. This is where everything is announced and commented
- ChildOfChaos 3y agoThanks, it's annoying some of the best places seem to be the places I am trying to avoid due to wanting to avoid the more negative aspects of it. Do you have any recommendations on twitter of who is good to follow? Particularly interested in people doing intreasting things with AI, I already subscribe to the normal AI newsletters, such as bens bites etc.
- screamingninja 3y agoWhat is a legitimate use case for this? I can think of a hundred applications for deceiving others but struggle to come up with a scenario where one would want their voice cloned or reproduced.
- smrtinsert 3y agoTiktok videos for the amusement of millions?
- dylan604 3y agoFor 6 months before it's banned!
- dannyw 3y agoYou're recording a podcast and want to tweak some of your own words, without the hassle of re-recording. You're an indie game developer, and want to have vibrant NPCs with their unique voices and dialogues powered by a LLM. You're producing a movie, and want to tweak certain lines of dialogue; with the consent of the talent. You suffer from health conditions and are gradually losing your voice, but you still want to communicate. There are certainly legitimate use cases of this technology. I personally believe illegitimate use cases overshadow the legitimate use cases, but I don't think it's fair to say there are no legitimate applications. We should strictly regulate the use of this technology by criminalizing abuse; not by banning it altogether (which is pretty hard in the case of software and small models).
- dylan604 3y ago> You're producing a movie, and want to tweak certain lines of dialogue; with the consent of the talent. The latest agreement to end the last round of strikes was to prevent this very thing. Of your list, the medical condition to give someone their real voice instead of a Hawking voice would be the most legit reason. Everything else is a skewed sense of morally acceptable as I think they are shady
- 3y ago
- grej 3y agoThis story just within the last days: "Athletic director used AI to frame principal with racist remarks in fake audio clip, police say" https://apnews.com/article/ai-artificial-intelligence-principal-audio-maryland-baltimore-county-pikesville-853ed171369bcbb888eb54f55195cb9c https://apnews.com/article/ai-artificial-intelligence-princi...
- skeaker 3y ago[deleted]
- fennecbutt 3y agoWhich is why we need to make this technology readily available and well known so that people are more aware of it and don't trust everything and look up sources. Ah, who am I kidding, most people will still not fact check.
- bparsons 3y agoYour scientists were so preoccupied with whether or not they could, they didn't stop to think if they should.
- deputy 3y ago[deleted]
- cryptonector 3y ago[flagged]
- tmaly 3y agoI see there are some python notebooks, but it would have been nice to see some example code in the README.
- skeaker 3y agoAwful lot of doomsaying and drama in here. What makes this release so bad compared to the existing voice cloning AI methods that have been publicly available for ~1 year already?