3 ms·
No, these are completely different, standalone code points, not variant forms of the same code point. What's actually happening seems to be that the ch_tra mod
by oefrha 6y ago
No, these are completely different, standalone code points, not variant forms of the same code point.
What's actually happening seems to be that the ch_tra model can recognize simplified too and output the corresponding traditional version if the character isn't in the traditional "alphabet"; it doesn't work so well in the other direction.
Example recognizing a partial screenshot of https://chinese.stackexchange.com/a/38707 https://chinese.stackexchange.com/a/38707 (anyone can try this on Google Colab, no hardware required; remember to turn on GPU in Runtime -> Change runtime type):
import easyocr
import requests
zhs_reader = easyocr.Reader(['en', 'ch_sim'])
zht_reader = easyocr.Reader(['en', 'ch_tra'])
image = requests.get('https://i.imgur.com/HtrpZCZ.png').content
print('ch_sim:', ' '.join(text for _, text, _ in zhs_reader.readtext(image)))
print('ch_tra:', ' '.join(text for _, text, _ in zht_reader.readtext(image)))
Results:
ch_sim: One simplified character may mapping to multiple traditional ones: 皇后->皇后,後夭->后夭 豌鬟->头发,骏财->发财 As reversed, one traditional character may mapping to multiple simplified ones too: 乾燥->干燥, 乾隆->乾隆 嘹望->嘹望,嘹解->了解
ch_tra: One simplified character may mapping to multiple traditional ones: 皇后->皇后,後天->后天 頭髮->頭發,發財->發財 As reversed, one traditional character may mapping to multiple simplified ones too: 乾燥->干燥, 乾隆->乾隆 瞭望->瞭望, 瞭解->了解
Compare to the original text:
One simplified character may mapping to multiple traditional ones:
- 皇后 -> 皇后,後天 -> 后天
- 頭髮 -> 头发,發財 -> 发财
As reversed, one traditional character may mapping to multiple simplified ones too:
- 乾燥 -> 干燥,乾隆 -> 乾隆
- 瞭望 -> 瞭望,瞭解 -> 了解
Of course, automatic character-to-character conversion from simplified to traditional can be wrong due to ambiguities; excellent examples from above: 头发 => 頭發 (should be 頭髮), 了解 => 了解 (should be 瞭解).
- jxy 6y agoFunny thing, 夭 and 天 are not the same at all. It doesn't seem to have a dictionary to do word level matching, only character level.
- oefrha 6y ago> Funny thing, 夭 and 天 are not the same at all. Yes, the simplified model is not that great at recognizing simplified either, at least in this case.
- divingdragon 6y agoThis approach seems a bit weird to me. While I appreciate them separating the models of Traditional and Simplified Chinese, I think I might prefer them to be combined (perhaps even including Japanese Kanji), and instead provide a way for the user to specify which language or regional variant is expected so characters matching the expected variant are simply given a higher score.
- oefrha 6y agoWithout delving into implementation details, I suspect the ch_tra model was simply trained on a dataset including simplified images with traditional labels.