4 ms·
Both Polish and English samples are actually synthesized with a voice trained on the WolneLektury audiobooks. They are the highest quality open source (CC BY-SA
by jpcl 3y ago
Both Polish and English samples are actually synthesized with a voice trained on the WolneLektury audiobooks. They are the highest quality open source (CC BY-SA) audiobooks I could find.
By using the Whisper-derived phonetic representation (so called semantic tokens) we successfully trained a model with just a high-quality speech dataset of one language and the voice quality transferred to English.
- satvikpendem 3y agoHow much training compute does it require to train from scratch? I'm wondering because I have a lot of audiobooks, they're not necessarily CC licensed though but for my private usage and training I think it'd be fine.
- jpcl 3y agoTraining the T2S model from scratch takes around 8h on 96 A100 GPUs. Training the `tiny` S2A model is around 3x faster (training HQ `small` variant is comparable to T2S). I think you would get good results with fine-tuning but unfortunately we don't have a user-friendly notebook or script to do that right now. The biggest model is 800MB (FP32) so you won't even need a very big GPU to be able to fine-tune.
- e12e 3y agoLink to these in English? I found some hits that may be correct for Polish - but I'm guessing they're hosted somewhere canonical?
- jpcl 3y agohttps://wolnelektury.pl/katalog/audiobooki/ https://wolnelektury.pl/katalog/audiobooki/ is the Polish audiobook collection. The English audiobooks are public domain recordings from LibriVox (via the LibriLight dataset).