2 ms·
I really just wish YouTube would detect captions embedded in their videos and stop displaying the same text (often incorrectly) on top of it. You do all this ma
by godelski 1y ago
I really just wish YouTube would detect captions embedded in their videos and stop displaying the same text (often incorrectly) on top of it. You do all this machine learning, why not put it into production? It's easy to cache the results and you're already scrubbing audio data and automatically doing STT, so extend it to do video to text and compare. It's not like this is an unsolved problem, even if imperfect. The audio provides a strong feedback for OCR errors
- amelius 1y agoI think the engineers at Google know and want this, but Google has a monopoly here so without a strong financial reason nothing will happen.
- yunusabd 1y agoSo you're saying that they should analyze both audio and video to increase the quality of the captions, if the video has hard-coded captions? I guess that's possible, just a question of effort vs. payoff. Inaccurate auto-captions for videos with hard coded captions probably isn't a big enough pain to warrant big investments?
- Leon_25 1y agoAgree. We build custom video analytics tools at Axon, and syncing OCR with audio-based STT isn’t rocket science anymore, especially with modern models. YouTube has all the ingredients, but seems slow to apply them at scale. Even basic alignment of audio captions with hardcoded subs would fix so much UX noise.