3 ms·

Certainly TTS has improved a lot thanks to modern AI, but it simply doesn't have the information to improve beyond sounding like a human reading words fluently.

Certainly TTS has improved a lot thanks to modern AI, but it simply doesn't have the information to improve beyond sounding like a human reading words fluently. A professional audiobook reader modulates his tone to reflect narrative mood, chooses voices for the characters consistent with their natures, etc., and transformer models can't do those things. For an example of a professional audiobook, check out Rob Inglis' version of The Lord of the Rings.

swores 1y ago
I agree with you about the current stage of things, which is why I said that we're approaching the point but not yet there for AI to be able to match professional readers. But I disagree with you when you write "it simply doesn't have the information to improve beyond sounding like a human reading words fluently" - it has the same information when reading it as a human does, meaning that the best implementation would have to not only adapt tone to explicit instructions like "... she shouted", but also read between the lines / make subjective choices to suit the different characters. AI is already capable of doing sentiment analysis on text, and text to speech models are getting better at being able to simulate moods/emotions rather than just speaking flatly, and I don't think we're many years away, if that, from those two sides being paired together in a way that produces the sort of quality output we're talking about for the first time without human involvement. Add to that the fact that AI can train on the many good examples of humans reading things, they may get to the point of emulating not just the core accent but also how each accent should adopt to what meanings in the text and arrive at a great solution without even needing to go through the steps of analysing what the text means to use that to know how to modify the voice being generated.
- username223 1y ago
  You're more optimistic about this stuff than I am, but I think I get your perspective. We have decent sentiment analysis, fluent text generation, and real-sounding TTS, so combining them will yield a pretty good reading. I agree that you're probably right when it comes to newspaper columns and magazine articles, but that's not on the level of a good audiobook. To take an example, here's an iconic line from the Fellowship of the Ring: > The wizard swayed on the bridge, stepped back a pace, and then again stood still. ‘You cannot pass!’ he said. If you think that is a command, you should shout it like Ian McKellen in the movie. If you think it's a statement based on superior knowledge (see https://acoup.blog/2025/04/25/collections-how-gandalf-proved-mightiest-spiritual-power-in-tolkien/ https://acoup.blog/2025/04/25/collections-how-gandalf-proved...), you should probably state it with certainty and fatigue. And if you're making a movie with a ton of crazy special effects and swelling music, you should probably make whatever choice goes best in that context. Even if a model could make some consistent choice there, I wouldn't be all that interested, because the reader conveying their interpretation of the character to the listener is what matters. Sure, it might get enough Spotify plays to make some money, but it's not art.

swores 1y ago

I agree with you about the current stage of things, which is why I said that we're approaching the point but not yet there for AI to be able to match professional readers. But I disagree with you when you write "it simply doesn't have the information to improve beyond sounding like a human reading words fluently" - it has the same information when reading it as a human does, meaning that the best implementation would have to not only adapt tone to explicit instructions like "... she shouted", but also read between the lines / make subjective choices to suit the different characters. AI is already capable of doing sentiment analysis on text, and text to speech models are getting better at being able to simulate moods/emotions rather than just speaking flatly, and I don't think we're many years away, if that, from those two sides being paired together in a way that produces the sort of quality output we're talking about for the first time without human involvement. Add to that the fact that AI can train on the many good examples of humans reading things, they may get to the point of emulating not just the core accent but also how each accent should adopt to what meanings in the text and arrive at a great solution without even needing to go through the steps of analysing what the text means to use that to know how to modify the voice being generated.

username223 1y ago
You're more optimistic about this stuff than I am, but I think I get your perspective. We have decent sentiment analysis, fluent text generation, and real-sounding TTS, so combining them will yield a pretty good reading. I agree that you're probably right when it comes to newspaper columns and magazine articles, but that's not on the level of a good audiobook. To take an example, here's an iconic line from the Fellowship of the Ring: > The wizard swayed on the bridge, stepped back a pace, and then again stood still. ‘You cannot pass!’ he said. If you think that is a command, you should shout it like Ian McKellen in the movie. If you think it's a statement based on superior knowledge (see https://acoup.blog/2025/04/25/collections-how-gandalf-proved-mightiest-spiritual-power-in-tolkien/ https://acoup.blog/2025/04/25/collections-how-gandalf-proved...), you should probably state it with certainty and fatigue. And if you're making a movie with a ton of crazy special effects and swelling music, you should probably make whatever choice goes best in that context. Even if a model could make some consistent choice there, I wouldn't be all that interested, because the reader conveying their interpretation of the character to the listener is what matters. Sure, it might get enough Spotify plays to make some money, but it's not art.