leobg

32 ms·

511.
by leobg 3y ago
Thank you!
512.
by leobg 3y ago
Thank you!
513.
by leobg 3y ago
They use pyannote/speaker-diarization. I tried that, but it wasn't accurate enough for my purposes. Made a confusion matrix with voice samples from The Simpsons characters. It looked... well, confused. Am using a mix now of speake
514.
by leobg 3y ago
Great website! You're based in France? You should put a demo on your website. If there is one, I don't see it (or rather: hear). If you're interested in collaborating, me email is in my bio.
515.
by leobg 3y ago
You could have asked the same question when Google started building their index. Or OpenAI trained their models. I'm in Germany. I'm a licensed lawyer. I see the dangers. The safest path will be to simply offer the production of m
516.
by leobg 3y ago
Perhaps you could ask some of your favorite podcast hosts to make a deal with me. Running a training on their voice once and then just re-using that will be much cheaper. Also, customers who buy in bulk will help me focus on this full time.
517.
by leobg 3y ago
FYI, I agree with you in all points. As I said in another comment, I wouldn't want to live in a world where everything was dubbed into my language. Any translation takes something away from the original. And dubbing even more so. I als
518.
by leobg 3y ago
Speakz actually generates subtitles as a byproduct. The idea is that you put in a video, you select the target languages. And then you get out, for each target language, an audio track and an SRT subtitles file. Someone else here asked abou
519.
by leobg 3y ago
Thank you. Exactly. I had the OG voice removed at first. But I added it back in for exactly this reason. It also serves as a tool for AI accountability: It lets you "see" that the cloned voice is indeed saying the same thing as th
520.
by leobg 3y ago
Yes! I love this! So you'd be translating English to Simplified English? Or are you talking from another source language? I've already been playing with this concept w.r.t. books: I take a non-fiction book. I'll have an LLM t
521.
by leobg 3y ago
Yes, in Germany we have generic voice-over narrators, too. In documentaries, etc.. They usually match the gender of the speaker, but that's it. Personally, I read most of my books with the iOS app Voice Dream Reader. That app still use
522.
by leobg 3y ago
Thank you very much! Yes. You cannot control the video playback on the demo page. I made it so because I wanted a way to showcase how you can switch between languages. You can go from Elon speaking English to German, Russian and Chinese, ea
523.
by leobg 3y ago
Thank you! With movies, I think I could get into legally challenging territory. I guess all AI apps are, in a way. But with movies, there's an entire industry behind enforcing copyright. So I must tread carefully on that front. I made
524.
by leobg 3y ago
Very good! Yes, that's basically how it works. I don't do any pitch-correction. But I do check the TTS output for lenght, and I re-generate if it doesn't match my time contraints. I also have an arranger that tries to figure
525.
by leobg 3y ago
Yes, that issue is also present in the German translation. I'm planning to monitor the output quality. Basically, feeding the translated audio back into the transcriber. Then compare it to the original transcript. Like a higher level l
526.
by leobg 3y ago
Yes. Though I don't like this solution. It breaks the flow. And it also doesn't really fully solve the problem. Overruns still accumulate if they happen too frequently. One second here, one second there... the further you get into
527.
by leobg 3y ago
Thanks! Well, to make local happen I'd have to learn more about local app development. I'd also be worried about having to support a bunch of different platforms, and being beholden to ever changing rules made by App Stores and OS
528.
by leobg 3y ago
One of my favorite movies was "Scent Of A Woman". But when I watched it in the German-dubbed version, I was appalled. It made the whole movie suddenly seem like a comedy. To me, the translation had killed its "soul", for
529.
by leobg 3y ago
I'm not sure. Making this work is limited by the availability by reliable transcription models. Which, in turn, are limited the availability of large training corpora. Those don't exist for rare languages. Also, if people choose t
530.
by leobg 3y ago
One idea I have is to use back-translation. After generating the new language audio, feed it back into the transcription, and then have an LLM compare it to the transcript from the original. Penalty for any thought/detail that is missi
531.
by leobg 3y ago
Not on my to do list currently. EzDubs say they do live [0]. Also, a friend of mine mentioned some Samsung / Android app that does this? [0] https://ezdubs.ai
532.
by leobg 3y ago
I don't have access to any special model that I'm holding back from you in order to rent-seek. Anyone can learn to build something like this. The parts are all available out there. There's Whisper. There's Mistral-7b. Th
533.
by leobg 3y ago
Let's just say this is NOT a wrapper around Elevenlabs or Heygen. I've looked at commercial voice cloning before. But, as you said, the prices seemed ridiculous. Before this, I made audiobooks for my daughter. Old, out-of-print bo
534.
by leobg 3y ago
Well, I'm also shamefully unoptimized at the moment. YouTube added auto-captions years ago. Long before there was Whisper, let alone things like Whisper.cpp. I imagine what I'm doing now is computationally no more expensive than w
535.
by leobg 3y ago
As a native German, I also resent it when Google/Amazon/whoever tries to force a translation on me when I prefer the original language. So I wouldn't want to live in a world where everything would be dubbed into German for me
536.
by leobg 3y ago
Thank you! Yeah, I also had a big grin on my face, hearing Lex and Elon suddenly talk in another language. :) Consistency isn't perfect yet, as we're building the voice from scratch basically for each utterance. One the one hand,
537.
by leobg 3y ago
I keep thinking about something similar. Also hardware. Also for my grandparents. My grandma is 95. Her vision is bad. Even using the phone (I'm talking old school landline) is getting hit and miss, because she can't see the butto
538.
by leobg 3y ago
I'm generating translated subtitles internally, before generating the voice-over. Also, generating those subtitles is way, way cheaper. If someone just wants the subtitles, I could offer them. Bigger question is: What device are your p
539.
by leobg 3y ago
Using a single speaker makes sense when you're paying for human voice talents. But if you're using a computer to generate the voice, why not generate a voice that sounds like the original speaker? Much more fun to hear Elon speak
540.
by leobg 3y ago
That's a great use case for LLMs, actually. Translate the sentence only up to what has been said so far. Basically, a balance between translating word-for-word (perfect timing, but terrible grammar) and translating the whole sentence a

More ›