8 ms·
> generated from text via a text-to-speech Yes, frustratingly we don't have good speech-to-text (STT/ASR) to transcribe such differences. I recently finetuned
by oezi 1y ago
> generated from text via a text-to-speech
Yes, frustratingly we don't have good speech-to-text (STT/ASR) to transcribe such differences.
I recently finetuned a TTS* to be able to emit laughter and hunting for transcriptions which include non-verbal sounds was the hardest part of it. Whisper and other popular transcription systems will ignore sigh, sniff, laugh, etc and can't detect mispronounciations etc.
* = https://github.com/coezbek/PlayDiffusion https://github.com/coezbek/PlayDiffusion
- jasonjayr 1y agoIIRC -- the 15.ai dev was training on fan-made "My Little Pony" transcriptions, specificaly because they included more emotive clues in the transcription, and supported a syntax to control the emotive aspect of the speech.
- dotancohen 1y agoWhere can I read about this?
- jasonjayr 1y ago> During this phase, 15 discovered the Pony Preservation Project, a collaborative project started by /mlp/, the My Little Pony board on 4chan.[47] Contributors of the project had manually trimmed, denoised, transcribed, and emotion-tagged thousands of voice lines from My Little Pony: Friendship Is Magic and had compiled them into a dataset that provided ideal training material for 15.ai.[48] From https://en.wikipedia.org/wiki/15.ai#2016%E2%80%932020:_Conception_and_development https://en.wikipedia.org/wiki/15.ai#2016%E2%80%932020:_Conce...
- vvolhejn 1y agoI had no idea this existed, the internet is amazing