3 ms·

> The reason is that I'm trying to collect data for learning languages, especially Chinese. Subtitles sites (opensubtitles, podnapisi, etc.) have similar data

> The reason is that I'm trying to collect data for learning languages, especially Chinese. Subtitles sites (opensubtitles, podnapisi, etc.) have similar data and simple alignment algorithms will work for unaligned subtitles. > Do the pirates really OCR the subtitles for their MKV files? OCR is really simple for text. No reason why they wouldn't do it. There were free programs decades ago for OCR when you were ripping DVDs and they worked mostly on character patterns (you would start with an empty database and then just tag the pattern, after a minute you'd have the whole subtitle OCR-ed).

peterburkimsher 7y ago
Please can you recommend a "simple alignment algorithm"? I can send some example ENG&CHT subtitles your way. There's a different number of subtitles, shown at different times, breaking lines in different places. It's a mess. If you know how to align it quickly, please tell me! Doing it manually takes me a couple of hours: longer than the whole movie. That's why I've only done it for a very few, specifically for movie nights with friends.
- superpermutat0r 7y ago
  You need good dictionaries, giving you all possible translations of a word both ways. Given that subtitles in both languages will have matching sentences very closely you can easily determine (be it some counting criteria, simple linear model or dynamic programming) which sequence of let's say 3 subtitle lines matches a sequence of 5 subtitle lines in the other translation. People/place/town names align stuff pretty easily. The fact that a sequence of lines matches some other sequence of lines makes finding the optimal alignment very efficient too. I'm guessing that just making a set of words for each subtitle line, counting the common words and picking a criteria of deciding if line maps to other line is more than enough. Subtitles are much easier than free-flowing documents because they are time constrained.
  - peterburkimsher 7y ago
    Please show me your best transcriptions for this channel [1]. There's over 2 TB of video data. No way to get access to the original PPT files. At least the subtitles on there are already aligned English & Chinese, but they do require OCR. Message me directly if you want my OpenCV script that can pull the yellow text out. Tesseract gave me this. The W0「d n! Gnd has bP〔‥mE ‥任Sh 嬰孩降生道咸肉員 Peace has Come for O… Km… 峒﹏m US 靜安來自‵我君手 ` I'm sorry to continue insisting, but seriously, Tesseract gave me terrible results. Identical lines don't even give the same OCR result, so trying to run partial matches on Google Translate-quality word equivalents isn't going to cut it. The OCR can't even give me the right number of characters! In the end I'm trying to do it manually, but it's really time-consuming. [1] https://www.youtube.com/channel/UC8_emPVKZvOwJnoBVpU0ytg/videos?view=0&sort=dd&shelf_id=1 https://www.youtube.com/channel/UC8_emPVKZvOwJnoBVpU0ytg/vid...
    - superpermutat0r 7y ago
      You should not use Tesseract for subtitles that blend with the colorful background. Some leptonica magic can extract that yellow stuff pretty easily. OpenCV is not as powerful for it, although it does have some leptonica functionality. I'll try to make a simple one in the next couple of hours. Although, the task you're asking me to do is different from subtitle alignment you could do on .srt files. What you can do with leptonica is extract the symbols (letters and ligatures) and manually tag them. You could also extract the lines containing letters and then process them through tesseract.
    - superpermutat0r 7y ago
      Poured out on the feet of Jesus 我們傾倒耶穌腳前 This is my output from tesseract (chi_tra) (for https://www.youtube.com/watch?v=RzIh2pamcwU https://www.youtube.com/watch?v=RzIh2pamcwU) I have just extracted the U component (YUV colorspace), binarized the image and tesseract just loves it. The players in the background are completely removed. I would probably use some background removal or some morphological operations from leptonica to make stuff more robust (instead of binarization). Of course, if tesseract does something incorrectly you can do a quick connected components in leptonica, extract the symbols and do your own OCR (manually). For chinese it's a bit of an issue because there are bunch of symbols, but for english you will be done tagging all the unique symbols in no time.
- okta 7y ago
  This tool https://github.com/kaegi/aligner https://github.com/kaegi/aligner worked well for EN-JP subs when I used it. The guy who wrote it describes the algorithm. Hope this helps.

peterburkimsher 7y ago

Please can you recommend a "simple alignment algorithm"? I can send some example ENG&CHT subtitles your way. There's a different number of subtitles, shown at different times, breaking lines in different places. It's a mess. If you know how to align it quickly, please tell me! Doing it manually takes me a couple of hours: longer than the whole movie. That's why I've only done it for a very few, specifically for movie nights with friends.

superpermutat0r 7y ago
You need good dictionaries, giving you all possible translations of a word both ways. Given that subtitles in both languages will have matching sentences very closely you can easily determine (be it some counting criteria, simple linear model or dynamic programming) which sequence of let's say 3 subtitle lines matches a sequence of 5 subtitle lines in the other translation. People/place/town names align stuff pretty easily. The fact that a sequence of lines matches some other sequence of lines makes finding the optimal alignment very efficient too. I'm guessing that just making a set of words for each subtitle line, counting the common words and picking a criteria of deciding if line maps to other line is more than enough. Subtitles are much easier than free-flowing documents because they are time constrained.
- peterburkimsher 7y ago
  Please show me your best transcriptions for this channel [1]. There's over 2 TB of video data. No way to get access to the original PPT files. At least the subtitles on there are already aligned English & Chinese, but they do require OCR. Message me directly if you want my OpenCV script that can pull the yellow text out. Tesseract gave me this. The W0「d n! Gnd has bP〔‥mE ‥任Sh 嬰孩降生道咸肉員 Peace has Come for O… Km… 峒﹏m US 靜安來自‵我君手 ` I'm sorry to continue insisting, but seriously, Tesseract gave me terrible results. Identical lines don't even give the same OCR result, so trying to run partial matches on Google Translate-quality word equivalents isn't going to cut it. The OCR can't even give me the right number of characters! In the end I'm trying to do it manually, but it's really time-consuming. [1] https://www.youtube.com/channel/UC8_emPVKZvOwJnoBVpU0ytg/videos?view=0&sort=dd&shelf_id=1 https://www.youtube.com/channel/UC8_emPVKZvOwJnoBVpU0ytg/vid...
  - superpermutat0r 7y ago
    You should not use Tesseract for subtitles that blend with the colorful background. Some leptonica magic can extract that yellow stuff pretty easily. OpenCV is not as powerful for it, although it does have some leptonica functionality. I'll try to make a simple one in the next couple of hours. Although, the task you're asking me to do is different from subtitle alignment you could do on .srt files. What you can do with leptonica is extract the symbols (letters and ligatures) and manually tag them. You could also extract the lines containing letters and then process them through tesseract.
  - superpermutat0r 7y ago
    Poured out on the feet of Jesus 我們傾倒耶穌腳前 This is my output from tesseract (chi_tra) (for https://www.youtube.com/watch?v=RzIh2pamcwU https://www.youtube.com/watch?v=RzIh2pamcwU) I have just extracted the U component (YUV colorspace), binarized the image and tesseract just loves it. The players in the background are completely removed. I would probably use some background removal or some morphological operations from leptonica to make stuff more robust (instead of binarization). Of course, if tesseract does something incorrectly you can do a quick connected components in leptonica, extract the symbols and do your own OCR (manually). For chinese it's a bit of an issue because there are bunch of symbols, but for english you will be done tagging all the unique symbols in no time.
okta 7y ago
This tool https://github.com/kaegi/aligner https://github.com/kaegi/aligner worked well for EN-JP subs when I used it. The guy who wrote it describes the algorithm. Hope this helps.