4 ms·

It is possible to synthesize an English voice with a 1.5MB model using http://cmuflite.org/ http://cmuflite.org/ or some of the Apple VoiceOver voices, which is

It is possible to synthesize an English voice with a 1.5MB model using http://cmuflite.org/ http://cmuflite.org/ or some of the Apple VoiceOver voices, which is just crazy to me. Most of the model is diphone samples for pairs of phonemes. I don't know of any way to go smaller than that with software. I tried, but it seems like a fundamental limit for English.

userbinator 2y ago
I don't know of any way to go smaller than that with software. I tried, but it seems like a fundamental limit for English. If you include "robotic" speech, then there's https://en.wikipedia.org/wiki/Software_Automatic_Mouth https://en.wikipedia.org/wiki/Software_Automatic_Mouth in a few tens of KB, and the demoscene has done similar in around 1/10th that. All formant synths, of course, not the sample-based ones that you're referring to.
- vidarh 2y ago
  SAM is awful but at the same time tantalisingly close (one of the demos of it apparently draws on a great reverse engineered version by Sebastian Macke and refactoring efforts by a couple of others including me - I spent too many hours listening to SAM output...) - especially when comparing to the still awful Festival/Flite models -, that I keep wanting to see what a better generic formant synth used as constraint on an ML model would produce. That is, instead of allowing a generic machine learning model to output unconstrained audio, train it on the basis of letting it produce low bitrate input/control values for a formant synth instead, and see just how small you can push the model.
  - tapper 2y ago
    Me as a blind dude that is listening to a sinth all the time I don't cair about the size of the thing I just want good SQ.
    - vidarh 2y ago
      That's fair, but those two are in constant tension. We can do very good speech if we ship gigs of samples. Better quality smaller models make it easier to get better quality speech in more places.
Animats 2y ago
Most of the links there are dead. That's a descendant of Festival Singer, which was well respected in its day. What's a current practical text-to-speech system that's open source, local, and not huge?
- follower 2y ago
  Depending on your definition of "huge", you might find Piper TTS fits your requirements: https://github.com/rhasspy/piper https://github.com/rhasspy/piper The size of the associated voice files varies but there are options that are under 100MB: https://huggingface.co/rhasspy/piper-voices/tree/main/en https://huggingface.co/rhasspy/piper-voices/tree/main/en
- anthk 2y ago
  Flite.
  - Animats 2y ago
    That's another spinoff of Festival Singer. What's good from the LLM era?
miki123211 2y ago
You probably could by what I call the "eastern-european method." Record one wave period of each phoneme, perhaps two for plosives, downsample to 8 or 11 kHz 8 bit, and repeat that recording on-the-fly enough times to make the right sound. If you're thinking "mod file", you're on the right track. For phonetically simple languages, such a system can easily fit on a microcontroller with kilobytes of RAM and a slow CPU. English might require a little bit more on the text-to-phoneme stage, but you can definitely go far below 1MB.
- rhdunn 2y ago
  That's effectively what eSpeak-ng is doing. For the CMU flite voices they represent the data as LPC (linear predictive coding) data with residual remainder (residual excited LPC). The HTS models use simple neural networks to predict the waveforms -- IIRC, these are similar to RNNs. The MBROLA models use OLA (overlapped add) to overlap small waveform samples. They also use diphone samples taken from midpoint to midpoint in order to create better phoneme transitions.
anthk 2y ago
I always wanted something like Flite but for Spanish.
aidenn0 2y ago
The speak & spell did it with a 32kB [edit, previously incorrectly wrote 16kB] ROM and a TMS0280.

userbinator 2y ago

I don't know of any way to go smaller than that with software. I tried, but it seems like a fundamental limit for English. If you include "robotic" speech, then there's https://en.wikipedia.org/wiki/Software_Automatic_Mouth https://en.wikipedia.org/wiki/Software_Automatic_Mouth in a few tens of KB, and the demoscene has done similar in around 1/10th that. All formant synths, of course, not the sample-based ones that you're referring to.

vidarh 2y ago
SAM is awful but at the same time tantalisingly close (one of the demos of it apparently draws on a great reverse engineered version by Sebastian Macke and refactoring efforts by a couple of others including me - I spent too many hours listening to SAM output...) - especially when comparing to the still awful Festival/Flite models -, that I keep wanting to see what a better generic formant synth used as constraint on an ML model would produce. That is, instead of allowing a generic machine learning model to output unconstrained audio, train it on the basis of letting it produce low bitrate input/control values for a formant synth instead, and see just how small you can push the model.
- tapper 2y ago
  Me as a blind dude that is listening to a sinth all the time I don't cair about the size of the thing I just want good SQ.
  - vidarh 2y ago
    That's fair, but those two are in constant tension. We can do very good speech if we ship gigs of samples. Better quality smaller models make it easier to get better quality speech in more places.

Animats 2y ago

Most of the links there are dead. That's a descendant of Festival Singer, which was well respected in its day. What's a current practical text-to-speech system that's open source, local, and not huge?

follower 2y ago
Depending on your definition of "huge", you might find Piper TTS fits your requirements: https://github.com/rhasspy/piper https://github.com/rhasspy/piper The size of the associated voice files varies but there are options that are under 100MB: https://huggingface.co/rhasspy/piper-voices/tree/main/en https://huggingface.co/rhasspy/piper-voices/tree/main/en
anthk 2y ago
Flite.
- Animats 2y ago
  That's another spinoff of Festival Singer. What's good from the LLM era?

miki123211 2y ago

You probably could by what I call the "eastern-european method." Record one wave period of each phoneme, perhaps two for plosives, downsample to 8 or 11 kHz 8 bit, and repeat that recording on-the-fly enough times to make the right sound. If you're thinking "mod file", you're on the right track. For phonetically simple languages, such a system can easily fit on a microcontroller with kilobytes of RAM and a slow CPU. English might require a little bit more on the text-to-phoneme stage, but you can definitely go far below 1MB.

rhdunn 2y ago
That's effectively what eSpeak-ng is doing. For the CMU flite voices they represent the data as LPC (linear predictive coding) data with residual remainder (residual excited LPC). The HTS models use simple neural networks to predict the waveforms -- IIRC, these are similar to RNNs. The MBROLA models use OLA (overlapped add) to overlap small waveform samples. They also use diphone samples taken from midpoint to midpoint in order to create better phoneme transitions.

anthk 2y ago

I always wanted something like Flite but for Spanish.

aidenn0 2y ago

The speak & spell did it with a 32kB [edit, previously incorrectly wrote 16kB] ROM and a TMS0280.