4 ms·

You need good dictionaries, giving you all possible translations of a word both ways. Given that subtitles in both languages will have matching sentences very c

You need good dictionaries, giving you all possible translations of a word both ways. Given that subtitles in both languages will have matching sentences very closely you can easily determine (be it some counting criteria, simple linear model or dynamic programming) which sequence of let's say 3 subtitle lines matches a sequence of 5 subtitle lines in the other translation. People/place/town names align stuff pretty easily. The fact that a sequence of lines matches some other sequence of lines makes finding the optimal alignment very efficient too. I'm guessing that just making a set of words for each subtitle line, counting the common words and picking a criteria of deciding if line maps to other line is more than enough. Subtitles are much easier than free-flowing documents because they are time constrained.

peterburkimsher 7y ago
Please show me your best transcriptions for this channel [1]. There's over 2 TB of video data. No way to get access to the original PPT files. At least the subtitles on there are already aligned English & Chinese, but they do require OCR. Message me directly if you want my OpenCV script that can pull the yellow text out. Tesseract gave me this. The W0「d n! Gnd has bP〔‥mE ‥任Sh 嬰孩降生道咸肉員 Peace has Come for O… Km… 峒﹏m US 靜安來自‵我君手 ` I'm sorry to continue insisting, but seriously, Tesseract gave me terrible results. Identical lines don't even give the same OCR result, so trying to run partial matches on Google Translate-quality word equivalents isn't going to cut it. The OCR can't even give me the right number of characters! In the end I'm trying to do it manually, but it's really time-consuming. [1] https://www.youtube.com/channel/UC8_emPVKZvOwJnoBVpU0ytg/videos?view=0&sort=dd&shelf_id=1 https://www.youtube.com/channel/UC8_emPVKZvOwJnoBVpU0ytg/vid...
- superpermutat0r 7y ago
  You should not use Tesseract for subtitles that blend with the colorful background. Some leptonica magic can extract that yellow stuff pretty easily. OpenCV is not as powerful for it, although it does have some leptonica functionality. I'll try to make a simple one in the next couple of hours. Although, the task you're asking me to do is different from subtitle alignment you could do on .srt files. What you can do with leptonica is extract the symbols (letters and ligatures) and manually tag them. You could also extract the lines containing letters and then process them through tesseract.
- superpermutat0r 7y ago
  Poured out on the feet of Jesus 我們傾倒耶穌腳前 This is my output from tesseract (chi_tra) (for https://www.youtube.com/watch?v=RzIh2pamcwU https://www.youtube.com/watch?v=RzIh2pamcwU) I have just extracted the U component (YUV colorspace), binarized the image and tesseract just loves it. The players in the background are completely removed. I would probably use some background removal or some morphological operations from leptonica to make stuff more robust (instead of binarization). Of course, if tesseract does something incorrectly you can do a quick connected components in leptonica, extract the symbols and do your own OCR (manually). For chinese it's a bit of an issue because there are bunch of symbols, but for english you will be done tagging all the unique symbols in no time.
  - peterburkimsher 7y ago
    That's amazing. Please can you tell me which version you're using? I tried tesseract 3.04.01 leptonica-1.74.1 libgif 4.2.3 : libjpeg 9c : libpng 1.6.34 : libtiff 4.0.9 : zlib 1.2.11 : libwebp 1.0.0 : libopenjp2 2.3.0 If you could email me your scripts, that would be extremely useful. And then I'll send you some subtitles where alignment is important. I just used this channel as an example of where I'd tried Tesseract and given up.

peterburkimsher 7y ago

Please show me your best transcriptions for this channel [1]. There's over 2 TB of video data. No way to get access to the original PPT files. At least the subtitles on there are already aligned English & Chinese, but they do require OCR. Message me directly if you want my OpenCV script that can pull the yellow text out. Tesseract gave me this. The W0「d n! Gnd has bP〔‥mE ‥任Sh 嬰孩降生道咸肉員 Peace has Come for O… Km… 峒﹏m US 靜安來自‵我君手 ` I'm sorry to continue insisting, but seriously, Tesseract gave me terrible results. Identical lines don't even give the same OCR result, so trying to run partial matches on Google Translate-quality word equivalents isn't going to cut it. The OCR can't even give me the right number of characters! In the end I'm trying to do it manually, but it's really time-consuming. [1] https://www.youtube.com/channel/UC8_emPVKZvOwJnoBVpU0ytg/videos?view=0&sort=dd&shelf_id=1 https://www.youtube.com/channel/UC8_emPVKZvOwJnoBVpU0ytg/vid...

superpermutat0r 7y ago
You should not use Tesseract for subtitles that blend with the colorful background. Some leptonica magic can extract that yellow stuff pretty easily. OpenCV is not as powerful for it, although it does have some leptonica functionality. I'll try to make a simple one in the next couple of hours. Although, the task you're asking me to do is different from subtitle alignment you could do on .srt files. What you can do with leptonica is extract the symbols (letters and ligatures) and manually tag them. You could also extract the lines containing letters and then process them through tesseract.
superpermutat0r 7y ago
Poured out on the feet of Jesus 我們傾倒耶穌腳前 This is my output from tesseract (chi_tra) (for https://www.youtube.com/watch?v=RzIh2pamcwU https://www.youtube.com/watch?v=RzIh2pamcwU) I have just extracted the U component (YUV colorspace), binarized the image and tesseract just loves it. The players in the background are completely removed. I would probably use some background removal or some morphological operations from leptonica to make stuff more robust (instead of binarization). Of course, if tesseract does something incorrectly you can do a quick connected components in leptonica, extract the symbols and do your own OCR (manually). For chinese it's a bit of an issue because there are bunch of symbols, but for english you will be done tagging all the unique symbols in no time.
- peterburkimsher 7y ago
  That's amazing. Please can you tell me which version you're using? I tried tesseract 3.04.01 leptonica-1.74.1 libgif 4.2.3 : libjpeg 9c : libpng 1.6.34 : libtiff 4.0.9 : zlib 1.2.11 : libwebp 1.0.0 : libopenjp2 2.3.0 If you could email me your scripts, that would be extremely useful. And then I'll send you some subtitles where alignment is important. I just used this channel as an example of where I'd tried Tesseract and given up.