4 ms·

Additionally when we speak we don't just 'replay' words; we add emotions and so on. Reminds me of this video [1] analysing how the actor Anthony Hopkins convert

• dkersten 10y ago
  I guess you could do something like vocaloid does to better "shape" the sounds of each word. I guess its pretty tough to do that automatically though, since words by themselves don't convey the emotions that they should be portrayed with. I mean, a word or phrase could very much change meaning depending on how its spoken (excitement, disgust, fear, sarcasm etc) and even if the context is available in the text (as is often the case), its rather difficult to machine-detect, at least, for now.
• angstrom 10y ago
  The spoken word has so many nuances. Cadence, inflexion, tone, and then the facial performances are never captured by pure text to speech. You'd almost have to be able to set boundaries for seriousness, playfulness, innuendo, indifference, sarcasm etc because none of that is conveyed in the literal text without additional descriptions or contextual analysis of the conversation.
  • derefr 10y ago
    It'd be interesting to take the Joint Many-Tasks approach here, training some of an RNN's layers on text-to-speech, and then another set of layers on sentiment analysis of speech-audio, where the error of the sentiment layer can backpropagate into the TTS layer.