23 ms·
Google's Street View computer vision can beat reCAPTCHA with 99% accuracy
- deleted 13y ago[deleted]
- adyus 13y agoIn effect, Google computer vision got so good that they made their own system obsolete. This is a good thing. I still think the only reliable way to confirm identity (or humanity) online is an email or SMS verification. Recently, receiving a 2-factor SMS code took less time than the page refresh prompting me to enter it.
- astrange 13y agoThat confirms identity, but it doesn't confirm humanity. Robots just need to have a SIM card! I think it would annoy many human users as well.
- duaneb 13y agoWe should just throw a standard turing test at them. "How was your day?"
- snewman 13y agoIt makes it much harder to flood a given site with a large number of actions. For instance, if you require SMS verification for account signup, and you don't allow the same phone number to be used for more than, say, two signups per month, then you make it hard to create a large number of bogus accounts. Yes, it will annoy some real users, so it's not a no-brainer. But it will put a major brake on many kinds of abuse.
- wpietri 13y agoYour underlying hypothesis here is that phone numbers are hard to get. But Twilio only charges $1/month per number, and presumably gets them for a lot less than that. If you use them for multiple nefarious purposes at once, your cost-per-evil could be pretty low. And that's assuming that you just don't steal them outright. Suppose you boost somebody's Android game, repackage it, and add a little code that intercepts and replies to certain text messages. It'd basically be a SMS botnet. SMS verification would make certain kinds of casual abuse harder, but I don't think it'd be a big barrier for well-organized assholes. E.g., the "Rachel from Cardholder Services" people that have been using the murkiness of the telephone network to sneak billions of illegal calls past the FTC.
- LaGrange 13y agoWhat you actually do if the verification is important enough (not often) is ask for a phone number, and have a human being call them. Some e-commerce sites do that, presumably ones exposed to frequent fraud, for example http://www.shatteredcrystal.com/ http://www.shatteredcrystal.com/, a game code reseller.
- Houshalter 13y agoProvide incentive to hack phones, discriminate against people who don't have phones or want to create new accounts.
- aroman 13y agoWouldn't it be trivial to code up a Twilio-powered bot to defeat the SMS verification?
- dfc 13y agoRequiring SMS or email creates problems for users that wish to remain anonymous. SMS is essentially not an option and the number of free email providers that accept registrations sans-SMS via tor continues to shrink.
- dlsym 13y ago"CAPTCHA images correctly solved by the algorithm" - Ok. Now I have to consider the possibility of being a machine.
- techaddict009 13y agoThis is really Great. AI is getting really smarter and smarter day by day!
- dnlbyl 13y ago99% is probably better than my success rate with reCAPTCHA...
- chrisBob 13y agoIn all seriousness, have there been any good studies of human success rates?
- sliverstorm 13y agoProbably better? I think my success rate is between 33-50%!
- yaddayadda 13y agoLooking at the four examples they give in the article, I'd likely only get one or two on the first go. - ntrtyLt (I'm mostly confident) - tmincrw (I'm really confident) - rrnmtht (I'm not at all confident of my answer, but I'm really confident that r's, n's, and m's should be prohibited next to each other in captchas like this!) - MCruncy (I'm really confident of my interpretation, but are capital letters supposed to be reported as capitals on this captcha system, or is this one of the systems that we're only supposed to submit lower case characters even when the character that we're viewing is upper case?)
- sajithdilshan 13y agoSkynet
- aaronbrethorst 13y agoI'm impressed that their address identification algorithm can solve those CAPTCHAs. I can't make heads or tails of them.
- deleted 13y ago[deleted]
- josho 13y agoInterestingly I activated a new gmail account today and during the signup process I experienced the obligatory captcha. It was in two parts, the first looked strikingly like a street view picture of a house number, while the second looked like a traditional captcha. I suspect that google has been using techniques like this to validate their computer vision conclusions. Which makes their 99% assertion even more interesting, because it's likely 99% confirmed by a very large crowd sourced data set, not simply a staff member going through several hundred samples to come up with the success rate.
- wmf 13y agoThat's been going on for two years: http://techcrunch.com/2012/03/29/google-now-using-recaptcha-to-decode-street-view-addresses/ http://techcrunch.com/2012/03/29/google-now-using-recaptcha-...
- 001sky 13y agoIsn't this like free mechanichal turk services? Or forced labour or something. Cynically brilliant outsourcing, tho.
- PavlovsCat 13y agoHumans recognize images, machine learning makes use of that in ways no human fully understands, and that makes it "google's computer vision"? AI has really come a long way :P http://en.wikipedia.org/wiki/The_Turk http://en.wikipedia.org/wiki/The_Turk
- vowelless 13y agoI highly recommend watching the TED talk by Luis von Ahn, creater of CAPTCHA and reCAPTCHA. He calls this "human computation". He is also the founder of Duolingo. https://www.youtube.com/watch?v=-Ht4qiDRZE8 https://www.youtube.com/watch?v=-Ht4qiDRZE8
- nathancahill 13y ago
- jere 13y ago>In this paper, we show that this system is able to accurately detect and read difficult numbers in Street View with 90% accuracy. > Turns out that this new algorithm can also be used to read CAPTCHA puzzles—we found that it can decipher the hardest distorted text puzzles from reCAPTCHA with over 99% accuracy. Am I missing something or could we improve CAPTCHAs by mimicking street numbers?
- Kronopath 13y agoreCAPTCHA already does this, actually. It's part of the way Google trains their computer vision algorithms to recognize numbers. http://techcrunch.com/2012/03/29/google-now-using-recaptcha-to-decode-street-view-addresses/ http://techcrunch.com/2012/03/29/google-now-using-recaptcha-...
- eurleif 13y agoBut that's, as you said, to train their computer vision, not to make the CAPTCHA better as a CAPTCHA.
- eurleif 13y ago>Am I missing something or could we improve CAPTCHAs by mimicking street numbers? Do the street numbers also have a lowered solve rate for humans?
- modeless 13y agoGoogle already uses actual street number images from Street View in reCAPTCHA. http://www.google.com/recaptcha/intro/ http://www.google.com/recaptcha/intro/
- daffodil2 13y agoWait, it's not clear to me from the blog post. Did they make a system that obsoletes reCAPTCHA? If so, it's just a matter of time before the spam systems catch up, correct? If so, what's the successor to CAPTCHA? Or is the web just going to be full of spam in the future?
- fuelfive 13y agoDigging in to the journal article, the technique they use can only scale to captchas with 8 characters or less, so having a longer word is a simple fix.
- zatkin 13y agoBut can it beat CRAPCHA? http://crapcha.com/ http://crapcha.com/
- Kequc 13y agoThe crapcha widget actually looks a lot better than reCaptcha. I've always lamented using reCaptcha just because of how ugly that thing is.
- maccard 13y agoDamn, that's better than me!
- mrsaint 13y agoCaptchas were meant to keep spammers at bay. Unfortunately, that's no longer the case. Thanks to "cloud technology" like DeathByCaptcha - that is, people in countries where labor is cheap solving captchas all day - spammers have no problem getting through reCaptcha-protected sites and forums to do their mischief. As a result, reCaptcha & co tend to be more of an annoyance to honest visitors than to spammers.
- Kequc 13y agoSounds like a bigger annoyance to spammers. I don't have to go out and enlist a captcha solving team in nigeria.
- t0lk 13y agoI have a mediawiki wiki and let me tell you there are tons of old spam bots out in the wild, ones that would fail at the presence of any captcha. Come to think of it, I've experienced most generations of spam bots since I started. With no protection I'd get spam links/registrations every day. With a simple question like 2+2=? I'd get one every 3-4 days or once a week and with reCaptcha I'd get them a little less frequently. I had to switch away from reCaptcha a few years back however when it appeared to no longer prevent spam (people are paid to solve them all day, etc).
- Keyframe 13y agoNow that programs are better and better at solving CAPTCHA - that means that correct CAPTCHA input will mean the opposite from what it means now. Since programs are better at solving CAPTCAH than humans, correct input (3/3 for example) will mean it's a robot. Thus, CAPTCHA becomes relevant again.
- sidko 13y agoAnd how would you deal with the fact that an algorithm could always intentionally provide the wrong answer? It's a losing game.
- LanceH 13y agoPresumably humans would see some pattern that isn't there. So you have to get the correct wrong result.
- wpietri 13y agoOoh, that's interesting. I was just talking with somebody yesterday about how his bosses had him spend six months trying to make a new financial system bug-compatible with an old financial system, one to which they had lost the source code. They eventually gave up and just kept using the old system so they could be consistent with previous publications. Making something bug-compatible with the human vision system might be much harder than pure recognition.
- LanceH 13y agoMy statement was mostly tongue in cheek. On the other hand, "what does that cloud look like?" would be an interesting captcha. Oh, and I might be better at the imagination ones than the current crop, especially on my phone.
- return0 13y agoThe endgame solution is to let humans judge other humans. In 5 years, you will be interrogated every time you post to HN. The machines are here to make our lives hell...
- zobzu 13y agoThe program solves captcha that I, as a human, cannot solve. Pretty sure that means captcha of that type are definitely dead.
- Blahah 13y agoSame here. Over the last few years captchas have been getting more and more difficult for me to decipher. It often takes me two attempts to get them right. Particularly annoying on google groups when you need to add a lot of members and you have to do a captcha for every ten members you add. Whatever replaces captcha, I really hope it's less frustrating for the actual humans
- jrochkind1 13y agoFrom that caption "CAPTCHA images correctly solved by the algorithm", there are at least two of them that I'm not sure _I_ can correctly solve on the first try. Which is generally my experience with captcha's these days, I only have about a 50% success rate. CAPTCHA is a failed strategy, time to give it up.
- pbhjpbhj 13y agoDiscussed this recently: https://news.ycombinator.com/item?id=7419667 https://news.ycombinator.com/item?id=7419667
- notatoad 13y agoit's a failed strategy for deterring spammers, but apparently reCAPTCHA has done a good job of training google's computer vision algos. which probably have better things to do than try to beat captchas.
- deleted 13y ago[deleted]
- pacofvf 13y agowell there are a lot of Resolve CAPTCHA as a Service sites like http://www.9kw.eu/ http://www.9kw.eu/
- pavelrub 13y agoThis is essentially the technology that was discussed here 3 months ago [1], and it links to the exact same article on arxiv, titled: "Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks". [2] They new addition to the article is that now they have tested the same type of NN on reCAPTCHA, and (perhaps unsurprisingly) it works. [1] - https://news.ycombinator.com/item?id=7015602 https://news.ycombinator.com/item?id=7015602 [2] - http://arxiv.org/abs/1312.6082v4 http://arxiv.org/abs/1312.6082v4.
- zwegner 13y agoThis particular issue (AI performance on captchas) is really quite fascinating. It's an arms race, but the problem is, only one side can win. Google is claiming they have improved their system in some (understandably) unspecified way, but there's only so far this can go. Captchas need to detect whether someone is human, but it has to work for everyone, ideally, even those with disabilities. Any simple task a human can do will eventually be able to be automated. Tasks that aren't currently feasible to be automated, say some natural language processing tasks, have another problem: scalability. To prevent simple databases of problems -> solutions, the problems need to be generated en masse, and for cheap, which means a computer needs to generate the solutions in the first place. And of course, paying people to just do captchas all day already happens. The street address/book scan approach that Google uses is interesting in that the exact solution is not known, so they presumably have to be somewhat forgiving in accepting answers (as their machine learning might have gotten it wrong). Perhaps this is what their "risk analysis" refers to--whether their response seems "human" enough according to their data, not necessarily whether it's correct. I don't see a way around this problem for free services that still preserves privacy (so directly using some government-issued ID is off the table). Maybe some Persona-like digital signature system, where a person can go to a physical location with a government ID, and get a signature that says "Trusted authority X affirms that person Y is in fact a person". Obviously this still has problems, as you need to trust X, and it's a big pain in the ass. There are parallels to the realm of passwords, which are also becoming obsolete (not that there's a good replacement...). Anything that a human can feasibly remember for a bunch of sites is becoming easier and easier for computers to guess. So basically, computers are taking over the world, and we can't do anything to stop it. God help us all.
- kevinchen 13y ago> The street address/book scan approach that Google uses is interesting in that the exact solution is not known, so they presumably have to be somewhat forgiving in accepting answers reCAPTCHA presents a known problem and an unknown problem to the user. If the user answers the known one correctly, it assumes that the answer to the unknown problem is correct too. I believe the text-based reCAPTCHAs will accept an answer within a character or two as correct.
- 13y ago
- leccine 13y agoAnd the cats, don't forget the cats!! http://www.wired.co.uk/news/archive/2012-06/26/google-brain-recognises-cats http://www.wired.co.uk/news/archive/2012-06/26/google-brain-...
- plg 13y agoWhy isn't google releasing the full algorithm?
- stuaxo 13y agoI'm sorry, as a human I have had to fill these street view style captchas in all the time for google, so this is hardly a completely artificial intelligence, humans have done it many many times, in fact I'm sure some of the pictures in the articles have come up.
- varunrau 13y agoI've always felt that it would be only a matter of time before computer vision would be able to solve the (re)CAPTCHA problem. Especially since digit classifiers are able to match the performance of humans. One approach that I enjoyed seeing was the use of reverse captchas. Here you pose a problem that a computer can easily solve, but a human cannot. For instance, if you ask a simple question (1+1=?), but you place the question box off the screen so the user can't see it. A computer would be able to easily answer the question, but a human user would have no way of doing so.
- drawkbox 13y ago99% is better than most humans captcha accuracy. Back in my day humans could still beat computers at Chess but nowadays computers can beat humans at Jeopardy and drive. Interesting to see when it fully crosses over.
- deleted 13y ago[deleted]
- northisup 13y agoYet it says I'm a robot a good two of three times.
- deleted 13y ago[deleted]
- conectorx 13y agothis is also can be done with tesseract or encog framework... i dont know whats news about this
- conectorx 13y agothis is also can be done with tesseract or encog framework... i dont know whats news about this
- EGreg 13y agoBasically consider why we want to eliminate computers from accessing the sites -- because we want to make account creation expensive, to prevent sybil attacks and giving away scarce resources. What is expensive? Reputation. That's where credit money's value comes from. I wrote a more comprehensive piece here, in case anyone's interested: https://news.ycombinator.com/item?id=7601690 https://news.ycombinator.com/item?id=7601690
- peterbotond 13y agowhat if someone has bad eyes of some rare eye problem and can not solve captcha problems at all? in other words fails captcha 90% of times.
- ilitirit 13y agoTo be honest, I can't even solve those reCAPTCHAs on that page (that's one of my biggest gripes about reCAPTCHA). I think we're nearing a point in time where if some(thing) can solve a particularly hard CAPTCHA, we can safely assume that it's not human.
- knodi 13y agoI just came here to say fuck reCAPTCHA! I hate it, I can't read it with my human eyes.
- ericcumbee 13y agoAnd you have added a lot to the conversation so thank you for that.
- frik 13y agoGoogle's reCAPTCHA showed street numbers as one of the two catcha-"words" for more than two years. For me this was quiet annoying to input street numbers of others. It's a privacy issue, it was like helping the NSA spying and one feels bad entering Google's captcha. What is even more astouning is that Google does not even mention all the croud sourced "volunteers" that trained their OCR database. As Google use an open OCR software (former HP OCR app from '95) it would be a good choice to publish their data back to the community. I removed Google captcha on my own sites and implemented my own traditional captcha (on the first sight of it about two years ago).
- aviraldg 13y agoIsn't this expected (and a natural consequence of the fact that it's trained on huge volumes of reCAPTCHA data?)
- spcoll 13y agoIt's a new success for Deep Learning. It seems to be actually 99.8% accuracy according to their paper: http://arxiv.org/abs/1312.6082 http://arxiv.org/abs/1312.6082 That's one order of magnitude higher.
- spullara 13y agoSo, now if you get the captcha right you're a computer, otherwise you are a human?
- spullara 13y agoReminds me of a hack day at Yahoo where one team made a captcha where you had to match a photo with its tags and another team made an algorithm that would assign tags to a photo. Both based on Flickr humans meant that the captcha was easily solvable by the algorithm.
- exadeci 13y agoYou're welcome google (we are the rats labs that teached their system how to read)
- blueskin_ 13y agoGreat... now they are going to get even harder to actually do.
- aljungberg 13y agoGoogle software could use their 99% successful algorithm to filter potential captchas. Then show the 1% they can't crack to humans. Now the race becomes who can write the better captcha solver, Google or the spammers? As spammers learn to identify things in the 1%, Google will hopefully improve faster and continue to narrow the "hard to solve" band.
- rasz_pl 13y ago>CAPTCHA images correctly solved by the algorithm well, isnt that great? Because I, HUMAN, can maybe solve _one_ of those (lower right one). I frickin HATE google Captchas and simply close the page if it wants me to solve one, they are too hard for me.
- infinity0 13y agoIronic how the HTTPS version force-redirects you to HTTP. (Amazon.co.uk started doing this a few days before and it's pissing me off no end.)
- rasz_pl 13y agoDoes google aggregate&correlate data in vision algo? For example for street numbers they not only have picture of a number, they also have knowledge of all the other numbers on that street and guesses for those other numbers. Easy to guesstimate order of a number by checking neighbouring ones. Same for book words, they have n-gram database. http://storage.googleapis.com/books/ngrams/books/datasetsv2.html http://storage.googleapis.com/books/ngrams/books/datasetsv2.... Thats a lot of useful MAP/ML data. But the example they give for the new captchas all look like random crap, "mhhfereeeem" and the like. Its like they are not interested in structure, just pure geometry of letters/numbers.
- msvan 13y agoHere's a captcha idea: make people write a 100-word essay on a specific topic. If it's good, you're accepted and you won't have to do it again. If it's bad, you're either a computer or cheap Nigerian labor. When we get to the point where we can't distinguish a computer from a human, we'll just let them be a part of the community.
- epi8 13y agoWho judges if the essay is good?
- vfclists 13y agoGoogle are getting too creepy for any sensible persons liking. Addresses which are off the street in apartment complexes are now getting recognized as well. Whenever I see these kind of captchas I switch to audio captchas. It is rather unethical for Google to use recaptchas in this way.
- tsenkov 13y agoIt's fascinating how, arguably simple software now, which is the captcha, would inevitably become more and more complex as AI develops.
- shultays 13y agoMy accuracy is way below 99%, good job Google! Seriously though, I hope this does not mean there will be harder captchas, current ones are already stupidly hard
- pestaa 13y agoFolks, I figured it out! Let's use captcha so that visitors can prove they are robots! If you fail these captchas, you must certainly be a human!
- scorpion032 12y agoThat is much more accuracy than with which humans can crack recaptcha. I'd estimate my own accuracy rate to be 90% or less.