8 ms·

On the one hand, this is very convenient. Probably cool for some non-fiction. On the other, some of my favorite audio books all stood out because the narrator

On the one hand, this is very convenient. Probably cool for some non-fiction. On the other, some of my favorite audio books all stood out because the narrator was interpreting the text really well, for example by changing the pacing during chaotic moments. Or those audiobooks with multiple narrators and different voices for each character. Not to mention that sometimes the only cue you get for who's speaking during dialogue is how the voice actor changes their tone. I have mixed feelings about using this and losing some of that quality. I would totally use this over amateur ebooks or public domain audiobooks like the ones on project guttenberg. As cool as it is/was for someone to contribute to free books... as a listener it was always jarring to switch to a new chapter and hear a completely different voice and microphone quality for no reason.

ahoka 2y ago
I guess this is still very useful if you are blind.
- loktarogar 2y ago
  Yeah, for accessibility purposes on things that aren't already narrated, this is kind of thing is huge.
  - em-bee 2y ago
    that's the thing. it's not just for accessibility. anything not already narrated is a fair target for TTS. i don't have time to sit down and read books. all reading is done on the go, while getting around or doing daily routines at home. i have a small book that i am reading now, which should take a few hours to finish, but in the time i manage to get done reading it i will probably have listened to two or three audio books. oh, and it's also a boon for those who can't afford to buy audiobooks.
    - vasco 2y ago
      You don't choose to spend your time reading books. You probably roll your eyes when someone tells you they don't have time for some activity you deem valuable. This is the 'no time to exercise' debate in a different shape. They are also different activities, with audio it's easier to listen to more but retention is usually lower. Not casting any elitist "you need to read" bullshit by the way, but find it odd to define it in terms of lack of time, and I really like both mediums.
      - em-bee 2y ago
        there is not much of a choice here. sure, i could use the time i spend reading and commenting on HN to read books instead. so technically speaking it is a choice. but i want to do both and many other things besides also having to work and a family to take care of. so the result is, i can't afford the time to read without giving up other things that are also important to me. listening to books allows me to access books i would otherwise not be able to read because of these priorities. there are other factors as well. i love reading so much that i tend to forget time around me. as a result reading would cause me to neglect other duties. i can't allow that, and therefore i am forced to avoid reading. i also don't like long form reading on electronic devices, and as a frequent traveler, printed books are simply not practical and often not even accessible. i agree with the retention issue, but i found that a much larger factor for retention is how well i can follow the story. a good story that is easy to get into is also easier to retain. and finally, reading fiction is for entertainment. i don't have to retain it.
      - esrauch 2y ago
        > You probably roll your eyes when someone tells you they don't have time for some activity you deem valuable. There's a few categories where it makes sense to roll your eyes, like if they say they have no time to shower or have never been to one of their kid's baseball games. But for things that aren't basic human expectations, I think you'd have to a real jerk to roll your eyes at someone not having time. No time to cook multi-pot dishes? No time to exercise? No time to read? No time to go to museums? No time to meet at the bar for a drink? Any of them sensible. No one can do everything, we all make our priorities and its well within their choice not to have any one optional life thing at the top of their personal stack.
    - loktarogar 2y ago
      Accessibility is generally framed around providing accomodations to people with disabilities, but at its core it's about more people being able to access things they otherwise couldn't. By this metric we agree
  - flir 2y ago
    I was just thinking about automatically slapping an mp3 on every blog post, just an accessibility nicety. Can someone with low vision tell me if this would be useful to them? It may be that specialist tools already do this better.
    - laserbeam 2y ago
      People use screen readers for accessibility. I would not expect anyone to be able to "look for and find" your mp3... I would instead expect them to use the tool they normally use for accessibility. The real question is "what tools are they already using and how can I make sure those tools are providing higher quality output?". There are standards in browsers for these kinds of things (ways to hint navigation via accessibility tools for example).
      - flir 2y ago
        > I would instead expect them to use the tool they normally use for accessibility. Yes, that was my second thought. But I'd rather ask someone than rely on my assumptions.
felixhummel 2y ago
I wholeheartedly agree. https://en.m.wikipedia.org/wiki/Stephen_Briggs https://en.m.wikipedia.org/wiki/Stephen_Briggs got me hooked on Terry Pratchett's Discworld series. I loved "Going Postal".
- IndrekR 2y ago
  I know someone who listened Terry Pratchett's "Wachen! Wachen!" audiobook on Spotify while living in Germany for few years. It was so well narrated that he also acquired some peculiarities of local dialects used by specific characters in the book. Locals in Bavaria were quite surprised of a foreigner speaking such language.
micw 2y ago
With this technology, one could produce high quality audio books without having access to high quality narrators by annotating the books with the voice, speed and such things. I wonder if a standardized markup exists to do so.
- KeplerBoy 2y ago
  Don't end to end trained models already do this to some extent? Like raising the pitch towards a question mark, like a human would. TortoiseTTS has a few examples under prompt engineering on their demo site: https://nonint.com/static/tortoise_v2_examples.html https://nonint.com/static/tortoise_v2_examples.html
  - micw 2y ago
    That's a bit of basic and random. Some models have the features you describe. From the better models you get a slightly different voice for text in quotes. But the difference to good audio books is that you have * different voices for the narrator and each character * different emotions and/or speed in certain situations. I guess you could use a LLM to "understand" and annotate an existing book if there's a markup and then use TTS to create an audio book from it and so automate most of the the process.
    - micw 2y ago
      Edit: I actually tried this. I prompted in ChatGPT: "Annotate the following text with speakers and emotions so that it can be turned into an audiobook via TTS", followed by a short text from "The Hobbit" (The "Good morning scene"). The result is very good.
- pegasus 2y ago
  They still wouldn't be high quality. It's just not possible to capture the precise tone of voice in an annotation, and that precision I believe really makes a difference. My experience is that the deeper the narrator understands the text and conveys that understanding, the easier it becomes for me to absorb that information.
stavros 2y ago
> On the other, some of my favorite audio books all stood out because the narrator was interpreting the text really well This (and everything else with AI) isn't saying "you don't need good actors any more". It's saying "if you don't have an audiobook, you can make a mediocre one automatically". AI (text, images, videos, whatever) doesn't replace the top end, it replaces the entire bottom-to-middle end.
- j4coh 2y ago
  RIP to future top-enders that would normally have started out on the bottom to middle end.
  - aredox 2y ago
    Bingo. AI is going to destroy any pathway for training and accruing experience. An embalming tech for our dying civilization.
    - lupusreal 2y ago
      Just like printing presses killed the profession of copying books by hand, eliminating the training pathway for illuminated manuscripts. Death of civilization itself I say, damn those printing presses.
      - j4coh 2y ago
        If you see podcasts as useless in modern society as illuminated manuscripts, no big loss I suppose, but I do enjoy the human made ones and would be sad to see them go extinct as the manuscripts did. And the same thing is happening to other entry-level creative roles, some of which you may personally regret the loss of too.
        akho 2y ago
        I enjoy looking at illuminated manuscripts. Podcasts are bullshit and can die in a ditch.
dmazin 2y ago
Absolutely. Even on the non-fiction side, the narration for Gleick's The Information adds something. While I want this tool for all the stuff with no narration, NYT/New Yorker/etc replacing human narrators with AI ones has been so shitty. The human narrators sound good, not just average. They add something. The AI narrators are simply bad.
Melomomololo 2y ago
I like one speaker in one particular book. He also narrates another scifi book series and honestly I dislike this a lot. He became the voice of one particular character for me. I would love variety
WillAdams 2y ago
Yes, but if the alternative is not having a book, or having to listen to one poorly read (I love Librivox, but there are some books which I just haven't been able to finish because of readers, and many more which were nixed for family vacation travel listening on that account), this may be workable.
rd11235 2y ago
I agree but the opposite can be true too. Sometimes the narrator seems to target some general audience that doesn’t fit me at all, in a way that makes me cringe when I listen, until I stop listening altogether. In these cases I’d rather listen to a relatively flat narration from a tool like this.
whazor 2y ago
A GenAI model that read audiobooks with such dramatisation is really my dream. There are so many books that I would want to listen to, but still lack such an adaptation. Also it takes months after the book release before the audiobook gets released. Just imagine what this would do for writers. They can get instant feedback and adjust their book for the audiobook.
ldoughty 2y ago
I agree with you, but also want to point out: New authors, self-publishers, can't afford tens of thousands of dollars to get an audiobook recorded professionally... This can limit their distribution. Authors might even choose not to make such version (or lack confidence to record themselves), so AI capable of making a decently passable version would be nice -- something more than reading text blandly. AI in theory could attempt to track the scene and adjust.
- DidYaWipe 2y ago
  You can get narrators to work on a royalty basis.
- plorg 2y ago
  By observation the current approach is for authors to narrate the book themselves of they think their readers will want it and if they feel reasonably confident in their own narration.
gmuslera 2y ago
Would a "better" AI would do a "better" narration with a better understanding of the text? Of course that it would imply a different (and far bigger?) model. Anyway, even if in theory it might, in practice things may end even worse than doing it with a monotone voice.
taude 2y ago
Agree with you on this. My example, I was never a Wheel of Time fan, but the new audio editions done by Rosamund Pike are quite the performance, and make me like the story. She brings all the characters to life in a way thats different than just reading. It's a true performance.
stevenwoo 2y ago
A couple of my favorite audiobooks are Stranger in a Strange Land and Flowers for Algernon where the performer changes the intonation and enunciation of main character with the character’s journey and it was a revelation and made me appreciate the stories in a way I did not get reading the printed books the first time. Just the consistency of the performance is sometimes difficult to do in my imagination perhaps.
Oneunscripted 2y ago
I guess using different narrators is essential for both fiction and non-fiction books if you want the full experience. Personally, I love it when audiobooks have narrators who stick to the characters’ personalities—it just feels right. Some of the audiobooks I’ve listened to have narrators who switch up their voices for each character, and others even use a different narrator for every character, which gets really good. Narration Box has been doing a really great job with this lately
lern_too_spel 2y ago
On the other hand, there are a lot of narrators who are just bad, and the publisher is not going to pay for an alternate narration. These tools are a good way to re-narrate Wil Wheaton narrated books with correct pronunciation and inflection, for example. Computer chess took a long time to get better than the best players in the world, but it was better than most chess players for many years before that. We're seeing that a lot with these generative models.

ahoka 2y ago

I guess this is still very useful if you are blind.

loktarogar 2y ago
Yeah, for accessibility purposes on things that aren't already narrated, this is kind of thing is huge.
- em-bee 2y ago
  that's the thing. it's not just for accessibility. anything not already narrated is a fair target for TTS. i don't have time to sit down and read books. all reading is done on the go, while getting around or doing daily routines at home. i have a small book that i am reading now, which should take a few hours to finish, but in the time i manage to get done reading it i will probably have listened to two or three audio books. oh, and it's also a boon for those who can't afford to buy audiobooks.
  - vasco 2y ago
    You don't choose to spend your time reading books. You probably roll your eyes when someone tells you they don't have time for some activity you deem valuable. This is the 'no time to exercise' debate in a different shape. They are also different activities, with audio it's easier to listen to more but retention is usually lower. Not casting any elitist "you need to read" bullshit by the way, but find it odd to define it in terms of lack of time, and I really like both mediums.
    - em-bee 2y ago
      there is not much of a choice here. sure, i could use the time i spend reading and commenting on HN to read books instead. so technically speaking it is a choice. but i want to do both and many other things besides also having to work and a family to take care of. so the result is, i can't afford the time to read without giving up other things that are also important to me. listening to books allows me to access books i would otherwise not be able to read because of these priorities. there are other factors as well. i love reading so much that i tend to forget time around me. as a result reading would cause me to neglect other duties. i can't allow that, and therefore i am forced to avoid reading. i also don't like long form reading on electronic devices, and as a frequent traveler, printed books are simply not practical and often not even accessible. i agree with the retention issue, but i found that a much larger factor for retention is how well i can follow the story. a good story that is easy to get into is also easier to retain. and finally, reading fiction is for entertainment. i don't have to retain it.
    - esrauch 2y ago
      > You probably roll your eyes when someone tells you they don't have time for some activity you deem valuable. There's a few categories where it makes sense to roll your eyes, like if they say they have no time to shower or have never been to one of their kid's baseball games. But for things that aren't basic human expectations, I think you'd have to a real jerk to roll your eyes at someone not having time. No time to cook multi-pot dishes? No time to exercise? No time to read? No time to go to museums? No time to meet at the bar for a drink? Any of them sensible. No one can do everything, we all make our priorities and its well within their choice not to have any one optional life thing at the top of their personal stack.
  - loktarogar 2y ago
    Accessibility is generally framed around providing accomodations to people with disabilities, but at its core it's about more people being able to access things they otherwise couldn't. By this metric we agree
- flir 2y ago
  I was just thinking about automatically slapping an mp3 on every blog post, just an accessibility nicety. Can someone with low vision tell me if this would be useful to them? It may be that specialist tools already do this better.
  - laserbeam 2y ago
    People use screen readers for accessibility. I would not expect anyone to be able to "look for and find" your mp3... I would instead expect them to use the tool they normally use for accessibility. The real question is "what tools are they already using and how can I make sure those tools are providing higher quality output?". There are standards in browsers for these kinds of things (ways to hint navigation via accessibility tools for example).
    - flir 2y ago
      > I would instead expect them to use the tool they normally use for accessibility. Yes, that was my second thought. But I'd rather ask someone than rely on my assumptions.

felixhummel 2y ago

I wholeheartedly agree. https://en.m.wikipedia.org/wiki/Stephen_Briggs https://en.m.wikipedia.org/wiki/Stephen_Briggs got me hooked on Terry Pratchett's Discworld series. I loved "Going Postal".

IndrekR 2y ago
I know someone who listened Terry Pratchett's "Wachen! Wachen!" audiobook on Spotify while living in Germany for few years. It was so well narrated that he also acquired some peculiarities of local dialects used by specific characters in the book. Locals in Bavaria were quite surprised of a foreigner speaking such language.

micw 2y ago

With this technology, one could produce high quality audio books without having access to high quality narrators by annotating the books with the voice, speed and such things. I wonder if a standardized markup exists to do so.

KeplerBoy 2y ago
Don't end to end trained models already do this to some extent? Like raising the pitch towards a question mark, like a human would. TortoiseTTS has a few examples under prompt engineering on their demo site: https://nonint.com/static/tortoise_v2_examples.html https://nonint.com/static/tortoise_v2_examples.html
- micw 2y ago
  That's a bit of basic and random. Some models have the features you describe. From the better models you get a slightly different voice for text in quotes. But the difference to good audio books is that you have * different voices for the narrator and each character * different emotions and/or speed in certain situations. I guess you could use a LLM to "understand" and annotate an existing book if there's a markup and then use TTS to create an audio book from it and so automate most of the the process.
  - micw 2y ago
    Edit: I actually tried this. I prompted in ChatGPT: "Annotate the following text with speakers and emotions so that it can be turned into an audiobook via TTS", followed by a short text from "The Hobbit" (The "Good morning scene"). The result is very good.
pegasus 2y ago
They still wouldn't be high quality. It's just not possible to capture the precise tone of voice in an annotation, and that precision I believe really makes a difference. My experience is that the deeper the narrator understands the text and conveys that understanding, the easier it becomes for me to absorb that information.

stavros 2y ago

> On the other, some of my favorite audio books all stood out because the narrator was interpreting the text really well This (and everything else with AI) isn't saying "you don't need good actors any more". It's saying "if you don't have an audiobook, you can make a mediocre one automatically". AI (text, images, videos, whatever) doesn't replace the top end, it replaces the entire bottom-to-middle end.

j4coh 2y ago
RIP to future top-enders that would normally have started out on the bottom to middle end.
- aredox 2y ago
  Bingo. AI is going to destroy any pathway for training and accruing experience. An embalming tech for our dying civilization.
  - lupusreal 2y ago
    Just like printing presses killed the profession of copying books by hand, eliminating the training pathway for illuminated manuscripts. Death of civilization itself I say, damn those printing presses.
    - j4coh 2y ago
      If you see podcasts as useless in modern society as illuminated manuscripts, no big loss I suppose, but I do enjoy the human made ones and would be sad to see them go extinct as the manuscripts did. And the same thing is happening to other entry-level creative roles, some of which you may personally regret the loss of too.
      - akho 2y ago
        I enjoy looking at illuminated manuscripts. Podcasts are bullshit and can die in a ditch.

dmazin 2y ago

Absolutely. Even on the non-fiction side, the narration for Gleick's The Information adds something. While I want this tool for all the stuff with no narration, NYT/New Yorker/etc replacing human narrators with AI ones has been so shitty. The human narrators sound good, not just average. They add something. The AI narrators are simply bad.

Melomomololo 2y ago

I like one speaker in one particular book. He also narrates another scifi book series and honestly I dislike this a lot. He became the voice of one particular character for me. I would love variety

WillAdams 2y ago

Yes, but if the alternative is not having a book, or having to listen to one poorly read (I love Librivox, but there are some books which I just haven't been able to finish because of readers, and many more which were nixed for family vacation travel listening on that account), this may be workable.

rd11235 2y ago

I agree but the opposite can be true too. Sometimes the narrator seems to target some general audience that doesn’t fit me at all, in a way that makes me cringe when I listen, until I stop listening altogether. In these cases I’d rather listen to a relatively flat narration from a tool like this.

whazor 2y ago

A GenAI model that read audiobooks with such dramatisation is really my dream. There are so many books that I would want to listen to, but still lack such an adaptation. Also it takes months after the book release before the audiobook gets released. Just imagine what this would do for writers. They can get instant feedback and adjust their book for the audiobook.

ldoughty 2y ago

I agree with you, but also want to point out: New authors, self-publishers, can't afford tens of thousands of dollars to get an audiobook recorded professionally... This can limit their distribution. Authors might even choose not to make such version (or lack confidence to record themselves), so AI capable of making a decently passable version would be nice -- something more than reading text blandly. AI in theory could attempt to track the scene and adjust.

DidYaWipe 2y ago
You can get narrators to work on a royalty basis.
plorg 2y ago
By observation the current approach is for authors to narrate the book themselves of they think their readers will want it and if they feel reasonably confident in their own narration.

gmuslera 2y ago

Would a "better" AI would do a "better" narration with a better understanding of the text? Of course that it would imply a different (and far bigger?) model. Anyway, even if in theory it might, in practice things may end even worse than doing it with a monotone voice.

taude 2y ago

Agree with you on this. My example, I was never a Wheel of Time fan, but the new audio editions done by Rosamund Pike are quite the performance, and make me like the story. She brings all the characters to life in a way thats different than just reading. It's a true performance.

stevenwoo 2y ago

A couple of my favorite audiobooks are Stranger in a Strange Land and Flowers for Algernon where the performer changes the intonation and enunciation of main character with the character’s journey and it was a revelation and made me appreciate the stories in a way I did not get reading the printed books the first time. Just the consistency of the performance is sometimes difficult to do in my imagination perhaps.

Oneunscripted 2y ago

I guess using different narrators is essential for both fiction and non-fiction books if you want the full experience. Personally, I love it when audiobooks have narrators who stick to the characters’ personalities—it just feels right. Some of the audiobooks I’ve listened to have narrators who switch up their voices for each character, and others even use a different narrator for every character, which gets really good. Narration Box has been doing a really great job with this lately

lern_too_spel 2y ago

On the other hand, there are a lot of narrators who are just bad, and the publisher is not going to pay for an alternate narration. These tools are a good way to re-narrate Wil Wheaton narrated books with correct pronunciation and inflection, for example. Computer chess took a long time to get better than the best players in the world, but it was better than most chess players for many years before that. We're seeing that a lot with these generative models.