3 ms·

Great questions! We include words from all different dialects, but standard Tokyo dialect will inevitably float to the top. I learned Japanese in Kansai, so I'

Great questions! We include words from all different dialects, but standard Tokyo dialect will inevitably float to the top. I learned Japanese in Kansai, so I'm sensitive to this issue. =) As for how the commonality is determined, I use a combination of corpuses including newspapers, novels, literature/poetry, and spoken language. I actually initially had a feature that would tell you "this word is more common in novels", but it didn't work consistently well enough, so I ended up scrapping it. When we actually choose how to order the search results, we take into account a few more things - like if the search string is part of the primary meaning of the word, if it's a partial match of the meaning string or a full match, etc. In order to determine the relative importances of the various factors, we actually automatically optimized them against a huge set of vocabulary lists that a Japanese native, English fluent speaker made for Japanese students studying English. This gives us the best chance of picking the Japanese word that a Japanese native speaker would choose for a given English word. As for the furigana, since we're taking arbitrary user input, there are bound to be mistakes here or there, and that's the case with this app too. I don't have any plans to let people edit the furigana right now, but I am improving the parsing all the time.

Nadya 11y ago
>we actually automatically optimized them against a huge set of vocabulary lists that a Japanese native, English fluent speaker made for Japanese students studying English. This gives us the best chance of picking the Japanese word that a Japanese native speaker would choose for a given English word. That's actually a pretty awesome way to handle it compared to other methods I've seen used. >As for the furigana, since we're taking arbitrary user input, there are bound to be mistakes here or there, and that's the case with this app too. Of course, I'm understanding of that issue. Parsing arbitrary input will always have errors, Google isn't exempt from this problem either. :) It's very, very difficult to get some of these things right (and sometimes just impossible given current technology/software). My issue with it is that not all people who use the app will be technologically savvy and understand that the furigana could be wrong. For someone learning a new language, giving the wrong reading could be extremely detrimental for them. I guess just voicing a gripe I have. I understand from a marketing point, every JP app does this when they automatically add furigana readings. I just wish there was some transparency that they aren't always accurate. :\
- chrisvasselli 11y ago
  Yeah, it's a good gripe to bring up. I'd love to come up with some indication of "confidence" in the parsing. Or maybe when there are two possible parses, give the user the ability to see both somehow?
  - Nadya 11y ago
    I think giving an option of two possible parsing would confuse people learning. They wouldn't know which is the right one to pick! So adding a second option just ads to the confusion... I think showing a confidence level would be a good solution based on the number of available readings for the given context of the kanji. 生 would be a good example. When used by itself, the confidence that the reading is correct should be low (since there are many possible options!) but if it is used in 生まれる the confidence level is very high (because there is only the one possible reading). When used in 生す it could be 50/50 (since it could be read な or む). Explaining this to learners is a little tricky.
    - chrisvasselli 11y ago
      Yeah, 生 is a good example. Fortunately, there are corpuses out there with hand-parsed sentences, so we at least train on those to pick the word for 生 on its own that's most common (I just tried - it's なま). We could use this to get our confidence even better than 50-50 for 生す. I can see how giving users more options could just bring more confusion. Such a tricky problem! I'll be thinking about this one.

Nadya 11y ago

>we actually automatically optimized them against a huge set of vocabulary lists that a Japanese native, English fluent speaker made for Japanese students studying English. This gives us the best chance of picking the Japanese word that a Japanese native speaker would choose for a given English word. That's actually a pretty awesome way to handle it compared to other methods I've seen used. >As for the furigana, since we're taking arbitrary user input, there are bound to be mistakes here or there, and that's the case with this app too. Of course, I'm understanding of that issue. Parsing arbitrary input will always have errors, Google isn't exempt from this problem either. :) It's very, very difficult to get some of these things right (and sometimes just impossible given current technology/software). My issue with it is that not all people who use the app will be technologically savvy and understand that the furigana could be wrong. For someone learning a new language, giving the wrong reading could be extremely detrimental for them. I guess just voicing a gripe I have. I understand from a marketing point, every JP app does this when they automatically add furigana readings. I just wish there was some transparency that they aren't always accurate. :\

chrisvasselli 11y ago
Yeah, it's a good gripe to bring up. I'd love to come up with some indication of "confidence" in the parsing. Or maybe when there are two possible parses, give the user the ability to see both somehow?
- Nadya 11y ago
  I think giving an option of two possible parsing would confuse people learning. They wouldn't know which is the right one to pick! So adding a second option just ads to the confusion... I think showing a confidence level would be a good solution based on the number of available readings for the given context of the kanji. 生 would be a good example. When used by itself, the confidence that the reading is correct should be low (since there are many possible options!) but if it is used in 生まれる the confidence level is very high (because there is only the one possible reading). When used in 生す it could be 50/50 (since it could be read な or む). Explaining this to learners is a little tricky.
  - chrisvasselli 11y ago
    Yeah, 生 is a good example. Fortunately, there are corpuses out there with hand-parsed sentences, so we at least train on those to pick the word for 生 on its own that's most common (I just tried - it's なま). We could use this to get our confidence even better than 50-50 for 生す. I can see how giving users more options could just bring more confusion. Such a tricky problem! I'll be thinking about this one.