9 ms·

Since audiobook enthusiasts might be in this thread ... does anyone know of a product (outside Kindle Whispersync) that can ingest an audiobook and the correspo

by ponker 6y ago

Since audiobook enthusiasts might be in this thread ... does anyone know of a product (outside Kindle Whispersync) that can ingest an audiobook and the corresponding e-book and using speech-to-text keep them roughly in sync so you can quickly go between the two?

gostsamo 6y ago
I'm not using it, but isn't this something like the stuff you are looking for? https://en.wikipedia.org/wiki/Digital_Accessible_Information_System https://en.wikipedia.org/wiki/Digital_Accessible_Information...
yorwba 6y ago
https://github.com/readbeyond/aeneas https://github.com/readbeyond/aeneas can output SMIL e-books where each line of text is associated with the corresponding segment from the audiobook. But I don't know which e-book readers support that format.
captn3m0 6y ago
I take my audible files, split them by chapter, and then manually jump to that chapter after checking in my audiobook player. Even syncing progress b/w ebook readers is a pain point for me right now[1], but I'd love to see solutions for this. [1]: https://github.com/johnfactotum/foliate/issues/584 https://github.com/johnfactotum/foliate/issues/584
dmitriid 6y ago
It's not a trivial task to implement. There are three stages: - Speech-to-text. It has to be good, or the text you get back is unusable. Google's and Amazon's services for that will quickly become expensive for large volumes. And you need to mark text with time/position if you want ebook-to-audiobok sync. - Diff algorithm between generated text and ebook text. Audiobooks are not produced from the same source material as ebooks, and they will have differences. On top of that you get errors and changes by narrators, and errors from speech-to-text. Not trivial if you want high fidelity. - Figuring out the position in ebook from the diff. Relatively simple. For ebook-to-audiobook you need to find the position in the speech-to-text conversion, and rewind to the time/position marking in that. Storytel (disclaimer: I work at Storytel) has quite a few ebooks and audiobooks connected, but doesn't allow you to ingest your own ebooks and audiobooks.
- switch11 6y ago
  interesting - read a lot of good things about you at the New Publishing blog that guy thinks you are going to be very big all the best. We sure need more people doing ebooks and audiobooks. Amazon is @#$#$ authors with all their exclusivity requirements and insanely high cut that they take
  - dmitriid 6y ago
    Thanks! We're trying! The whole audiobook landscape isn't clear cut yet. In music and video you have a few well-established (entrenched?) players: Amazon, Google, Apple, Spotify (music). In audiobooks there are still a bunch of smaller players alongside Amazon (Apple doesn't do much about audiobooks, other big players are only just starting, see Spotify). There are at least three in every country, I think :) And the whole thing is further fractured by whether or not publishers and rights holders want to provide their audiobook, and ebooks, or only one or the other. And there are way more publishers and rights holders than there are in music (in music they are also mostly consolidated into a few megamonsters [1]). And the market is still growing, as more and more people discover and listen audiobooks (and related material like podcasts) during commute, in the car, during daily routines and so on. So yeah, it's stressful, but interesting :) [1] https://en.wikipedia.org/wiki/Record_label#3 https://en.wikipedia.org/wiki/Record_label#3
- ponker 6y ago
  You wouldn’t need good speech to text. The purpose would only be to establish synchronization between audio and text, and where you don’t have synchronization you could assume that the speaker is carrying on at their normal words per minute.
  - dmitriid 6y ago
    > The purpose would only be to establish synchronization between audio and text To establish synchronisation you need to have, well, two texts and a way to compare them ;) > where you don’t have synchronization you could assume that the speaker is carrying on at their normal words per minute. What is their normal WPM and how would you calculate that? ;)

by ponker 6y ago

gostsamo 6y ago

I'm not using it, but isn't this something like the stuff you are looking for? https://en.wikipedia.org/wiki/Digital_Accessible_Information_System https://en.wikipedia.org/wiki/Digital_Accessible_Information...

yorwba 6y ago

https://github.com/readbeyond/aeneas https://github.com/readbeyond/aeneas can output SMIL e-books where each line of text is associated with the corresponding segment from the audiobook. But I don't know which e-book readers support that format.

captn3m0 6y ago

I take my audible files, split them by chapter, and then manually jump to that chapter after checking in my audiobook player. Even syncing progress b/w ebook readers is a pain point for me right now[1], but I'd love to see solutions for this. [1]: https://github.com/johnfactotum/foliate/issues/584 https://github.com/johnfactotum/foliate/issues/584

dmitriid 6y ago

It's not a trivial task to implement. There are three stages: - Speech-to-text. It has to be good, or the text you get back is unusable. Google's and Amazon's services for that will quickly become expensive for large volumes. And you need to mark text with time/position if you want ebook-to-audiobok sync. - Diff algorithm between generated text and ebook text. Audiobooks are not produced from the same source material as ebooks, and they will have differences. On top of that you get errors and changes by narrators, and errors from speech-to-text. Not trivial if you want high fidelity. - Figuring out the position in ebook from the diff. Relatively simple. For ebook-to-audiobook you need to find the position in the speech-to-text conversion, and rewind to the time/position marking in that. Storytel (disclaimer: I work at Storytel) has quite a few ebooks and audiobooks connected, but doesn't allow you to ingest your own ebooks and audiobooks.

switch11 6y ago
interesting - read a lot of good things about you at the New Publishing blog that guy thinks you are going to be very big all the best. We sure need more people doing ebooks and audiobooks. Amazon is @#$#$ authors with all their exclusivity requirements and insanely high cut that they take
- dmitriid 6y ago
  Thanks! We're trying! The whole audiobook landscape isn't clear cut yet. In music and video you have a few well-established (entrenched?) players: Amazon, Google, Apple, Spotify (music). In audiobooks there are still a bunch of smaller players alongside Amazon (Apple doesn't do much about audiobooks, other big players are only just starting, see Spotify). There are at least three in every country, I think :) And the whole thing is further fractured by whether or not publishers and rights holders want to provide their audiobook, and ebooks, or only one or the other. And there are way more publishers and rights holders than there are in music (in music they are also mostly consolidated into a few megamonsters [1]). And the market is still growing, as more and more people discover and listen audiobooks (and related material like podcasts) during commute, in the car, during daily routines and so on. So yeah, it's stressful, but interesting :) [1] https://en.wikipedia.org/wiki/Record_label#3 https://en.wikipedia.org/wiki/Record_label#3
ponker 6y ago
You wouldn’t need good speech to text. The purpose would only be to establish synchronization between audio and text, and where you don’t have synchronization you could assume that the speaker is carrying on at their normal words per minute.
- dmitriid 6y ago
  > The purpose would only be to establish synchronization between audio and text To establish synchronisation you need to have, well, two texts and a way to compare them ;) > where you don’t have synchronization you could assume that the speaker is carrying on at their normal words per minute. What is their normal WPM and how would you calculate that? ;)