4 ms·
This is a great idea; I really enjoy all these "two channels simultaneously" (side-by-side translations, video with subtitles, and in this case video with a rea
by svat 4y ago
This is a great idea; I really enjoy all these "two channels simultaneously" (side-by-side translations, video with subtitles, and in this case video with a readable transcript, where you can scroll in the video or scroll in the transcript, and be synchronized).
I had done something like this a couple of years ago for some specific set of videos (e.g. https://shreevatsa.net/tex/program/videos/s10/ https://shreevatsa.net/tex/program/videos/s10/ — compare with https://youtubetranscript.com/?v=_0Cv1G_s4gQ https://youtubetranscript.com/?v=_0Cv1G_s4gQ for the same video), but never got around to making it general; glad someone has done it. It takes just a few lines of Javascript, using the Youtube API, to do this i.e. keeping the video and text in sync (just view source on either page to see the JS at the bottom).
Something like this can also help with audio recordings (generating the alignment automatically is called "forced alignment" and there are tools like "aeneas" for this). In case anyone's interested or wants to help (for Sanskrit texts): see https://github.com/shreevatsa/web-align-audio-text https://github.com/shreevatsa/web-align-audio-text deployed at https://shreevatsa.net/ramayana/sarga/ https://shreevatsa.net/ramayana/sarga/ and better version at https://github.com/avinashvarna/audio_alignment https://github.com/avinashvarna/audio_alignment deployed at https://avinashvarna.github.io/audio_alignment/ https://avinashvarna.github.io/audio_alignment/
- BasilPH 4y agoThis is cool! We're doing forced alignment with audio recordings i.e. podcasts. Here's an example from a test client: https://www.withfanfare.com/p/seldon-crisis/search-by-the-foundation-part-iii https://www.withfanfare.com/p/seldon-crisis/search-by-the-fo... Grateful for any feedback you might have. Also, if you run a knowledge-dense podcast, or know somebody that does, I would love to talk to you/them. I'm for example considering to link places, people and things in a transcript.
- svat 4y agoLovely, thanks for the example. I really like how the underline is subtle yet effective. The animation/effect where it seems to inch forward is also intriguing; how does that work? (Also, this is subjective but have you considered making double-clicking on the text open the audio player even when it's not been started yet? Would make it more discoverable, though I imagine some could get annoyed. Another minor thing: when the audio is paused and you double click somewhere, the audio position changes but this is not reflected in the player; only takes effect when you unpause.) You're doing cool work, good luck with your service! It looks very useful and if someone I know is running a podcast I'll recommend it for sure; it looks really well-done and polished (at least the example you shared).
- BasilPH 4y agoThank you! I wrote your points down. We're still learning how to make it most usable. We know the start and end timestamps on a word level and we know the current player time. So all we do is set a CSS class on the words that are currently playing. We not only highlight the current word, but also the words that are close in time. This is what generates the effect. We just released autoscroll, which is something that was requested often.
- marcelfahle 4y agoWow, this looks incredible! Love how the underline travels and guides the eye.. Very impressive! Excuse the question, but "forced alignment" is when you don't have timestamps, like in webvtt?
- BasilPH 4y agoThank you! Yes, exactly. We do forced alignment when you edit your transcript. The new words don't have any timestamps, so we need to align them. For short sections we use interpolation. If we need align whole sections we use Gentle[^1]. [^1]: https://github.com/lowerquality/gentle https://github.com/lowerquality/gentle
- marcelfahle 4y agoThank you, this is really interesting! :)
- deleted 4y ago[deleted]