5 ms·

How far away are we from AI reading books so well that everybody who has the book in text form automatically has it in audio form, because they can just turn it

by mg 3y ago

How far away are we from AI reading books so well that everybody who has the book in text form automatically has it in audio form, because they can just turn it into audio via a multitude of AI text to speech networks? By the way, I would love a service where I could listen to HN. Play the first title and then wait for me to hit the space bar to play the next. Highlight the one I just heard, so when I'm interested int a submission, I have it right in front of me.

sxg 3y ago
We're already there. I use Readwise Reader and often take advantage of the text-to-speech feature for listening to articles from the Economist. The Economist already provides an audio narration of the article by a human, but some of the narrators' British accents are a bit too thick and difficult for me to understand at 1.5x. Readwise Reader's text-to-speech does a much better job at 1.5x for my American English brain. AI-narration isn't perfect—here are some issues I have encountered: - Section headers are read just like regular body text, which is confusing - Not great with reading numbers. Narrating a year is very different from narrating a monetary value. - Parenthetical statements are read just like regular body text
- mmahemoff 3y ago
  I also use Readwise Reader and the speech update they dropped a month ago made great strides towards AI reading. It’s just missing some UX improvements now, mainly the ability to continue playing across multiple articles. It’s also not smart enough to skip boilerplate junk that appears on top and bottom of every newsletter, which is easy to skip over visually but a pain when read out loud. A while ago, Amazon tried adding TTS to kindle and got immediately slammed by the publishers, so they withdrew it. However, with the current state of AI, I think users will start to demand it as a standard feature of anything they’re reading. Not only can it read at human-like capability, it can do so in any voice theoretically. Soon you’ll also be able to pause it and ask questions about what it just read or to summarize the rest of the article and so on.
  - elektor 3y ago
    I just tried out Readwise Reader's speech feature and it looks like they use Unreal Speech[1]. The "Liv" voice sounded pretty realistic. 1: https://unrealspeech.com/pricing https://unrealspeech.com/pricing
impostervt 3y ago
KDP, the part of amazon that lets you publish your own books, now has a beta program to create audio books using virtual voices. https://www.kdpcommunity.com/s/article/Invite-Only-KDP-Beta-for-Audiobooks?language=en_US&forum=KDP%20Forum https://www.kdpcommunity.com/s/article/Invite-Only-KDP-Beta-...
- jjice 3y ago
  Neat feature. I assume is AWS Polly under the hood. This caught my eye though: > Authors can set a list price between $3.99 and $14.99 and will receive a 40% royalty Do they mean you get 40% of the work you created and they ran their model on? Oh lord. Let me run this through a TTS myself and publish it like that. Not sure if they even allow that though.
  - impostervt 3y ago
    That's the standard payment when publishing your audiobook using ACX (how independents get their books on Audible) - and then only if you go exclusive with Audible. If you're not exclusive, you get 25%. Audible's market share is like 60%, so you loose out on a lot if you don't publish there.
intpx 3y ago
https://techcrunch.com/2022/06/13/spotify-is-acquiring-sonantic-the-ai-voice-platform-used-to-simulate-val-kilmers-voice-in-top-gun-maverick/ https://techcrunch.com/2022/06/13/spotify-is-acquiring-sonan... Before the acquisition, sonatics tech looked pretty cool. https://techcrunch.com/2022/06/13/spotify-is-acquiring-sonantic-the-ai-voice-platform-used-to-simulate-val-kilmers-voice-in-top-gun-maverick/ https://techcrunch.com/2022/06/13/spotify-is-acquiring-sonan... that being said-- NOT A FAN. i want books read by human people who are good at reading to me. I dont want teddy ruxpin, i want my dad!
swashboon 3y ago
Audible does not allow for AI narration of Public Domain books currently.
fassssst 3y ago
It’s already there, some of the books I’m listening to on Libby are clearly AI.
aedocw 3y ago
I wrote https://github.com/aedocw/epub2tts https://github.com/aedocw/epub2tts and the quality is pretty phenomenal if you use XTTS for the voice generation. It's really excellent, but requires the book not have DRM.
FredPret 3y ago
You can use iOS’s built-in screen reader to narrate the Kindle app. I listened to an entire book that way. Audible is much better but you can do it this way too.
- ascagnel_ 3y ago
  Hardware Kindles (and the app) used to support TTS for accessibility, but the functionality was removed at the request of publishers (who wanted to sell audiobooks).
- lostlogin 3y ago
  This is pretty great. A colleague was saying to me that they would like to be able to have an ebook to read, but also to have it read out loud when driving or walking.
  - FredPret 3y ago
    I listened to an entire Kindle book this way many years ago. I imagine / hope Siri's screenreading only got better in that time. I think you need to set the Kindle app to continuous scrolling (in which case it won't work with older books), then you trigger the iOS screenreading shortcut, and off it goes.
crazygringo 3y ago
On the one hand, the technology is already there, if you're merely looking for a "mechanical" rendering that sounds like a person with no emotion or understanding of the text -- no meaningful emphases, no meaningful change in rhythm and pitch, etc. (Well and the occasional mispronunciation of a word, especially heteronyms). But if you're looking for tech that can read it as well as a human can in terms of turning meaning into prosody -- all of the vocal inflections that we use and hear without even realizing it, the things that make it sound human and appropriate to the text -- that's hard to say. (Especially when you consider that most people are bad at audiobook narration, as it's a real skill.) It might be very soon, if the kind of meaning embedded in LLM's turns out to be perfect for the job. Or it might be very far away, if it turns out that it requires building a kind of accurate model of human emotion and feeling and how that maps to vocal qualities, that has nothing to do with how LLM's work.
- mongol 3y ago
  Apple has already done it: https://www.theguardian.com/technology/2023/jan/04/apple-artificial-intelligence-ai-audiobooks https://www.theguardian.com/technology/2023/jan/04/apple-art...
  - crazygringo 3y ago
    Yes, that's exactly the kind of "mechanical" version I was talking about. The article goes on to describe the challenges I was talking about: > Still, computer-generated voices have long struggled to hold the attention of listeners for long periods of time, and to overcome the ‘uncanny valley’ effect of synthetic human speech. Human intonation and inflection are notoriously difficult to predict and replicate.
    - deleted 3y ago
      [deleted]
  - burkaman 3y ago
    Some samples here: https://authors.apple.com/support/4519-digital-narration-audiobooks https://authors.apple.com/support/4519-digital-narration-aud.... It's good, but it would definitely get very monotonous to listen to for 10+ hours, and even in these short cherry-picked examples there are a couple kind of jarring tonal decisions. You'll also notice they avoided sampling any dialogue, presumably because these voices will read all characters in the exact same manner as the narration. I listen to a lot of audiobooks and I definitely notice and get annoyed when I don't like the narrator. I think I could maybe get used to this for one book but I would avoid it if possible, and I would really hate it if every book I listened to used this exact same style. I would say this is probably already good enough for short non-fiction articles, but not there yet for fiction. It is genuinely confusing (for me at least) when an audiobook narrator reads a phrase with the "wrong" emotion, or doesn't vocally distinguish between characters.

by mg 3y ago

sxg 3y ago

We're already there. I use Readwise Reader and often take advantage of the text-to-speech feature for listening to articles from the Economist. The Economist already provides an audio narration of the article by a human, but some of the narrators' British accents are a bit too thick and difficult for me to understand at 1.5x. Readwise Reader's text-to-speech does a much better job at 1.5x for my American English brain. AI-narration isn't perfect—here are some issues I have encountered: - Section headers are read just like regular body text, which is confusing - Not great with reading numbers. Narrating a year is very different from narrating a monetary value. - Parenthetical statements are read just like regular body text

mmahemoff 3y ago
I also use Readwise Reader and the speech update they dropped a month ago made great strides towards AI reading. It’s just missing some UX improvements now, mainly the ability to continue playing across multiple articles. It’s also not smart enough to skip boilerplate junk that appears on top and bottom of every newsletter, which is easy to skip over visually but a pain when read out loud. A while ago, Amazon tried adding TTS to kindle and got immediately slammed by the publishers, so they withdrew it. However, with the current state of AI, I think users will start to demand it as a standard feature of anything they’re reading. Not only can it read at human-like capability, it can do so in any voice theoretically. Soon you’ll also be able to pause it and ask questions about what it just read or to summarize the rest of the article and so on.
- elektor 3y ago
  I just tried out Readwise Reader's speech feature and it looks like they use Unreal Speech[1]. The "Liv" voice sounded pretty realistic. 1: https://unrealspeech.com/pricing https://unrealspeech.com/pricing

impostervt 3y ago

KDP, the part of amazon that lets you publish your own books, now has a beta program to create audio books using virtual voices. https://www.kdpcommunity.com/s/article/Invite-Only-KDP-Beta-for-Audiobooks?language=en_US&forum=KDP%20Forum https://www.kdpcommunity.com/s/article/Invite-Only-KDP-Beta-...

jjice 3y ago
Neat feature. I assume is AWS Polly under the hood. This caught my eye though: > Authors can set a list price between $3.99 and $14.99 and will receive a 40% royalty Do they mean you get 40% of the work you created and they ran their model on? Oh lord. Let me run this through a TTS myself and publish it like that. Not sure if they even allow that though.
- impostervt 3y ago
  That's the standard payment when publishing your audiobook using ACX (how independents get their books on Audible) - and then only if you go exclusive with Audible. If you're not exclusive, you get 25%. Audible's market share is like 60%, so you loose out on a lot if you don't publish there.

intpx 3y ago

https://techcrunch.com/2022/06/13/spotify-is-acquiring-sonantic-the-ai-voice-platform-used-to-simulate-val-kilmers-voice-in-top-gun-maverick/ https://techcrunch.com/2022/06/13/spotify-is-acquiring-sonan... Before the acquisition, sonatics tech looked pretty cool. https://techcrunch.com/2022/06/13/spotify-is-acquiring-sonantic-the-ai-voice-platform-used-to-simulate-val-kilmers-voice-in-top-gun-maverick/ https://techcrunch.com/2022/06/13/spotify-is-acquiring-sonan... that being said-- NOT A FAN. i want books read by human people who are good at reading to me. I dont want teddy ruxpin, i want my dad!

swashboon 3y ago

Audible does not allow for AI narration of Public Domain books currently.

fassssst 3y ago

It’s already there, some of the books I’m listening to on Libby are clearly AI.

aedocw 3y ago

I wrote https://github.com/aedocw/epub2tts https://github.com/aedocw/epub2tts and the quality is pretty phenomenal if you use XTTS for the voice generation. It's really excellent, but requires the book not have DRM.

FredPret 3y ago

You can use iOS’s built-in screen reader to narrate the Kindle app. I listened to an entire book that way. Audible is much better but you can do it this way too.

ascagnel_ 3y ago
Hardware Kindles (and the app) used to support TTS for accessibility, but the functionality was removed at the request of publishers (who wanted to sell audiobooks).
lostlogin 3y ago
This is pretty great. A colleague was saying to me that they would like to be able to have an ebook to read, but also to have it read out loud when driving or walking.
- FredPret 3y ago
  I listened to an entire Kindle book this way many years ago. I imagine / hope Siri's screenreading only got better in that time. I think you need to set the Kindle app to continuous scrolling (in which case it won't work with older books), then you trigger the iOS screenreading shortcut, and off it goes.

crazygringo 3y ago

On the one hand, the technology is already there, if you're merely looking for a "mechanical" rendering that sounds like a person with no emotion or understanding of the text -- no meaningful emphases, no meaningful change in rhythm and pitch, etc. (Well and the occasional mispronunciation of a word, especially heteronyms). But if you're looking for tech that can read it as well as a human can in terms of turning meaning into prosody -- all of the vocal inflections that we use and hear without even realizing it, the things that make it sound human and appropriate to the text -- that's hard to say. (Especially when you consider that most people are bad at audiobook narration, as it's a real skill.) It might be very soon, if the kind of meaning embedded in LLM's turns out to be perfect for the job. Or it might be very far away, if it turns out that it requires building a kind of accurate model of human emotion and feeling and how that maps to vocal qualities, that has nothing to do with how LLM's work.

mongol 3y ago
Apple has already done it: https://www.theguardian.com/technology/2023/jan/04/apple-artificial-intelligence-ai-audiobooks https://www.theguardian.com/technology/2023/jan/04/apple-art...
- crazygringo 3y ago
  Yes, that's exactly the kind of "mechanical" version I was talking about. The article goes on to describe the challenges I was talking about: > Still, computer-generated voices have long struggled to hold the attention of listeners for long periods of time, and to overcome the ‘uncanny valley’ effect of synthetic human speech. Human intonation and inflection are notoriously difficult to predict and replicate.
  - deleted 3y ago
    [deleted]
- burkaman 3y ago
  Some samples here: https://authors.apple.com/support/4519-digital-narration-audiobooks https://authors.apple.com/support/4519-digital-narration-aud.... It's good, but it would definitely get very monotonous to listen to for 10+ hours, and even in these short cherry-picked examples there are a couple kind of jarring tonal decisions. You'll also notice they avoided sampling any dialogue, presumably because these voices will read all characters in the exact same manner as the narration. I listen to a lot of audiobooks and I definitely notice and get annoyed when I don't like the narrator. I think I could maybe get used to this for one book but I would avoid it if possible, and I would really hate it if every book I listened to used this exact same style. I would say this is probably already good enough for short non-fiction articles, but not there yet for fiction. It is genuinely confusing (for me at least) when an audiobook narrator reads a phrase with the "wrong" emotion, or doesn't vocally distinguish between characters.