4 ms·

Word Lens for Google Glass

89 points by jf 13y ago

tobyjsullivan 13y ago
I like the idea of this. I'm curious how well it'll actually work in practice. Will the Glass camera require you to hold everything up to your face?
- dag11 13y ago
  Actually, since the camera currently has no ability to focus (fixed to infinity), it only works well for large type far away. It works best with large signage.
  - hnriot 13y ago
    in my experience, the type of text that meets these criteria is nearly always in english, or doesn't need translation - like street signs, airport signs, bill boards etc. Translation should really be "lookup in my language" - I should be able to lookup with something on a menu says, and what it means, for example. This is where things like Word Lens fails, it just does a word-for-word translation. Take hors d'oeuvres, as an example, translates (word for word) as "out work", which is very unhelpful. There are times when i'd like something that looks at a MWE and tells me what it is, no matter what language it was written in. Like Google does.
  - otaviogood 13y ago
    Yes, it works best with larger fonts. There's not a lot we can do about the focus on the software side unfortunately.
andrewljohnson 13y ago
1) I remember the original post. Word Lens lit up #1 on Hacker News for an impressive amount of time: https://news.ycombinator.com/item?id=2014555 https://news.ycombinator.com/item?id=2014555 2) Looks like jf did the original submission too, I wonder if he knows a Word Lens founder. 3) I like the forward-looking embrace of Google Glass. It's such a futuristic thing to have shades that translate for you. Your phone is essentially a babelfish already - how long before you can buy an auto-translating bluetooth headset?
- peteforde 13y ago
  So what if jf is a founder of WordLens? Congratulations to them.
  - deleted 13y ago
    [deleted]
- antonioevans 13y ago
  What if wordlens is purchased by google and incorporated into glass. stop being so dev-pessimistic (we need a word for this).
- veemjeem 13y ago
  jf does know the word lens founder, but then many people here in the bay area also know Octavio -- he's a super cool "always positive" person. You'd probably like him if you met him.
  - andrewljohnson 13y ago
    I don't mean it in a harsh way, I just noted it because I noticed. When I saw this submission, I tried to Google to find the original post. When I couldn't find it, I thought to click the OP's name, to see if they had submitted the link. If the word lens founder told jf to post this, that doesn't bother me, and I don't consider either party unethical. It's just interesting to think about how social news works.
    - jf 13y ago
      Otavio didn't know about HN when he released Word Lens, so I posted it here for him and made him make an account so he could reply to people in the comments. Ever since then, he's asked me to announce their new products here for him, so I do ...
      - hnriot 13y ago
        how does a san francisco game developer not know of hn...
seliopou 13y ago
I forgot that Google Glass existed. I'm not saying that to be smart. I actually read the headline and caught myself in the middle of asking myself, What's Google Glass? Am I the only one, or what? I guess people just haven't been talking about it lately or something. Weird.
- dannyr 13y ago
  Actually, it is possible that something exists even if you don't know or talk about it. /sarcasm
physcab 13y ago
I actually might buy Google Glass just to have this app when I travel. It's the perfect app for Glass. I just hope Google can take away the auditory cues of "Ok glass, do this...". I should be able to just turn on Glass, steady my head, and get the translation.
- bhups 13y ago
  You already can. For those that have Glass, if you wake the screen, it gets you to "ok glass", tap again, scroll through the options until you get to "Translate". Edit: More generally, any "Ok Glass..." action can be triggered this way.
swalsh 13y ago
Oh man, this is great. I have the World lens app for the ipad, i didn't have an iphone when I was in France... so I was "that guy" holding an ipad up in public. My wife tried to distance herself from me :D The app is not perfect, it does a word for word translation, so sometimes its a bit weird. However most of the time its good enough, and it is surprisingly flexible with different signs. It definitely saved me from eating some things i'm just not adventurous enough to try :D
mrinterweb 13y ago
Word Lens is amazing software, but from my experience with it, its results are frequently more confusing than trying to decipher the foreign language on your own. Word Lens shines when you are using it on text that is large, on a plain background, in good lighting, and a recognized font. If one or more of those conditions are not met, your results will likely be less than satisfactory. For example, trying to use Word Lens to read most restaurant menus is usually not productive, in my experience. I love the spirit of the software and what it is trying to accomplish. I just don't feel that it is yet useful for translating most of the text you may be trying to read. A good companion to Word Lens is the Google Translate app. I found the Google Translate app was much more accurate when I took pictures of text that I wanted translated. The downside to the Google Translate app is that it requires a data connection which can be tricky when traveling in another country.
- eru 13y ago
  On Android you can cache the whole `dictionary'. It takes in the order of 1 GiB per language pair.
  - mrinterweb 13y ago
    That is fantastic. I'll need to try that. I wish I knew that a couple weeks ago.
- hnriot 13y ago
  "amazing software" - yet "usually not very productive" - I agree with you. Google (visual) translate is both built on better statistical foundations and works better (likely as a result of the math and the gobs of data google have to refine with.)
jeremydw 13y ago
I just downloaded it to Glass and tried it out. The UI is nice, and very simple. You say "OK Glass, translate this" and then you point your face (the camera) at some text (there's a bounding box on the display that you put the text into). You need to hold your head still, and then the software expands the bounding box to the full size of the screen. Everything I've described so far happens very quickly, and then slowly some of the words on the screen are translated and replaced in place. The lighting in my room is not very bright, and so far I've been unable to get full sentences to translate (just words, and just for a moment until I move my head a little), but I suspect it'd work better in high contrast situations (out in the sun, block text, etc.) Tapping the side of Glass lets you switch between to/from English/Spanish. Edit: Last thing, IMO, the largest downside to the UX right now is that when you move your face (even a tiny bit) the in-place translation disappears and you need to hold your face steady again to get the translation to appear. This would obviously make it pretty difficult to use in a real world situation (like, if you're walking outside – you'd need to stop and hold very still to translate a sign) but it's still super cool.
- otaviogood 13y ago
  Word Lens needs a lot of light. If there isn't enough light, the scene will have noise and motion blur. That's bad for the OCR algorithms. The camera also makes a huge difference. As cameras get more sensitive, the noise and motion blur are reduced. That has been our experience working with Word Lens over the years.
vlad 13y ago
This must have been the app I saw at Super Happy Dev House in 2009 or 2010, because it was secret at the time. I remember the creator was a really nice guy and this is a great idea.
- otaviogood 13y ago
  Yes, this is the same app I was showing to people at Super Happy Dev House pretty much the whole way through the development process. I'm a big fan of SHDH. So many creative people.
danellis 13y ago
I feel like I'm missing something obvious here, but how do I actually get this?

89 points by jf 13y ago

I like the idea of this. I'm curious how well it'll actually work in practice. Will the Glass camera require you to hold everything up to your face?

dag11 13y ago
Actually, since the camera currently has no ability to focus (fixed to infinity), it only works well for large type far away. It works best with large signage.
- hnriot 13y ago
  in my experience, the type of text that meets these criteria is nearly always in english, or doesn't need translation - like street signs, airport signs, bill boards etc. Translation should really be "lookup in my language" - I should be able to lookup with something on a menu says, and what it means, for example. This is where things like Word Lens fails, it just does a word-for-word translation. Take hors d'oeuvres, as an example, translates (word for word) as "out work", which is very unhelpful. There are times when i'd like something that looks at a MWE and tells me what it is, no matter what language it was written in. Like Google does.
- otaviogood 13y ago
  Yes, it works best with larger fonts. There's not a lot we can do about the focus on the software side unfortunately.

andrewljohnson 13y ago

1) I remember the original post. Word Lens lit up #1 on Hacker News for an impressive amount of time: https://news.ycombinator.com/item?id=2014555 https://news.ycombinator.com/item?id=2014555 2) Looks like jf did the original submission too, I wonder if he knows a Word Lens founder. 3) I like the forward-looking embrace of Google Glass. It's such a futuristic thing to have shades that translate for you. Your phone is essentially a babelfish already - how long before you can buy an auto-translating bluetooth headset?

peteforde 13y ago
So what if jf is a founder of WordLens? Congratulations to them.
- deleted 13y ago
  [deleted]
antonioevans 13y ago
What if wordlens is purchased by google and incorporated into glass. stop being so dev-pessimistic (we need a word for this).
veemjeem 13y ago
jf does know the word lens founder, but then many people here in the bay area also know Octavio -- he's a super cool "always positive" person. You'd probably like him if you met him.
- andrewljohnson 13y ago
  I don't mean it in a harsh way, I just noted it because I noticed. When I saw this submission, I tried to Google to find the original post. When I couldn't find it, I thought to click the OP's name, to see if they had submitted the link. If the word lens founder told jf to post this, that doesn't bother me, and I don't consider either party unethical. It's just interesting to think about how social news works.
  - jf 13y ago
    Otavio didn't know about HN when he released Word Lens, so I posted it here for him and made him make an account so he could reply to people in the comments. Ever since then, he's asked me to announce their new products here for him, so I do ...
    - hnriot 13y ago
      how does a san francisco game developer not know of hn...

seliopou 13y ago

I forgot that Google Glass existed. I'm not saying that to be smart. I actually read the headline and caught myself in the middle of asking myself, What's Google Glass? Am I the only one, or what? I guess people just haven't been talking about it lately or something. Weird.

dannyr 13y ago
Actually, it is possible that something exists even if you don't know or talk about it. /sarcasm

physcab 13y ago

I actually might buy Google Glass just to have this app when I travel. It's the perfect app for Glass. I just hope Google can take away the auditory cues of "Ok glass, do this...". I should be able to just turn on Glass, steady my head, and get the translation.

bhups 13y ago
You already can. For those that have Glass, if you wake the screen, it gets you to "ok glass", tap again, scroll through the options until you get to "Translate". Edit: More generally, any "Ok Glass..." action can be triggered this way.

swalsh 13y ago

Oh man, this is great. I have the World lens app for the ipad, i didn't have an iphone when I was in France... so I was "that guy" holding an ipad up in public. My wife tried to distance herself from me :D The app is not perfect, it does a word for word translation, so sometimes its a bit weird. However most of the time its good enough, and it is surprisingly flexible with different signs. It definitely saved me from eating some things i'm just not adventurous enough to try :D

mrinterweb 13y ago

Word Lens is amazing software, but from my experience with it, its results are frequently more confusing than trying to decipher the foreign language on your own. Word Lens shines when you are using it on text that is large, on a plain background, in good lighting, and a recognized font. If one or more of those conditions are not met, your results will likely be less than satisfactory. For example, trying to use Word Lens to read most restaurant menus is usually not productive, in my experience. I love the spirit of the software and what it is trying to accomplish. I just don't feel that it is yet useful for translating most of the text you may be trying to read. A good companion to Word Lens is the Google Translate app. I found the Google Translate app was much more accurate when I took pictures of text that I wanted translated. The downside to the Google Translate app is that it requires a data connection which can be tricky when traveling in another country.

eru 13y ago
On Android you can cache the whole `dictionary'. It takes in the order of 1 GiB per language pair.
- mrinterweb 13y ago
  That is fantastic. I'll need to try that. I wish I knew that a couple weeks ago.
hnriot 13y ago
"amazing software" - yet "usually not very productive" - I agree with you. Google (visual) translate is both built on better statistical foundations and works better (likely as a result of the math and the gobs of data google have to refine with.)

jeremydw 13y ago

I just downloaded it to Glass and tried it out. The UI is nice, and very simple. You say "OK Glass, translate this" and then you point your face (the camera) at some text (there's a bounding box on the display that you put the text into). You need to hold your head still, and then the software expands the bounding box to the full size of the screen. Everything I've described so far happens very quickly, and then slowly some of the words on the screen are translated and replaced in place. The lighting in my room is not very bright, and so far I've been unable to get full sentences to translate (just words, and just for a moment until I move my head a little), but I suspect it'd work better in high contrast situations (out in the sun, block text, etc.) Tapping the side of Glass lets you switch between to/from English/Spanish. Edit: Last thing, IMO, the largest downside to the UX right now is that when you move your face (even a tiny bit) the in-place translation disappears and you need to hold your face steady again to get the translation to appear. This would obviously make it pretty difficult to use in a real world situation (like, if you're walking outside – you'd need to stop and hold very still to translate a sign) but it's still super cool.

otaviogood 13y ago
Word Lens needs a lot of light. If there isn't enough light, the scene will have noise and motion blur. That's bad for the OCR algorithms. The camera also makes a huge difference. As cameras get more sensitive, the noise and motion blur are reduced. That has been our experience working with Word Lens over the years.

vlad 13y ago

This must have been the app I saw at Super Happy Dev House in 2009 or 2010, because it was secret at the time. I remember the creator was a really nice guy and this is a great idea.

otaviogood 13y ago
Yes, this is the same app I was showing to people at Super Happy Dev House pretty much the whole way through the development process. I'm a big fan of SHDH. So many creative people.

danellis 13y ago

I feel like I'm missing something obvious here, but how do I actually get this?