9 ms·
CAPTCHAs are supposed to tell humans and robots apart, if robots are all doing better than humans, maybe we should flip the acceptance criteria to make sure you
by bfeynman 3y ago
CAPTCHAs are supposed to tell humans and robots apart, if robots are all doing better than humans, maybe we should flip the acceptance criteria to make sure you are not performing task at superhuman level, (until we train bots to do this). On an unrelated note, I have found captchas that don't even work and reprompt me all the time, I wonder if there is some naive filtering behavior they are applying.
- camel-cdr 3y agoI feel like they are sometimes already doing that, I had multiple CAPTCHAs where I thought I miss clicked, but got through anyways.
- microtonal 3y agoIt’s been like that for years, they probably use a subset for extra annotation.
- hunter2_ 3y agoI think only some parts of a CAPTCHA are challenging you (data is labeled sufficiently for mistakes to be considered a failed solve), and the other parts are still in the process of being labeled, which you are helping to accomplish, so those have no influence on the immediate outcome. At least that's how the old "type these two words" CAPTCHAs worked. It was crowdsourced human OCR of whatever text the machine OCR couldn't make sense of. I'm not sure if "find the bus/motorcycle/crosswalk/light" is the same way, but perhaps, and it does seem to offer leeway when there's only a few pixels of that item in the frame.
- jorvi 3y agoWhat’s sad is, I’ve started to predict how dumb other humans are at filling in CAPTCHA’s. Often there’s a piece of motorcycle, stair, traffic light, fire hydrant, or non-bus picture that users will mis-select, and I’ll do what I think the masses do. I used to choose correctly but being sent through 5 chains of CAPTCHAs is modern hell.
- calfuris 3y agoI'm not certain that that's what is happening, but I strongly suspect it. It is if nothing else a useful mental model: switching my strategy from selecting every square that contains foo to selecting every square that I think that Joe Schmoe would select gets me through CAPTCHAs more consistently.
- chromoblob 3y ago> until we train bots to do this This is trivial.
- zer0w1re 3y agoI always thought that's what captchas evolved into anyways. Human-reinforced training for AI image recognition.
- HankB99 3y agoThat seems to square with the content I'm frequently tested in. Crosswalks and traffic signs for self driving cars and stairs for walking robots. I've also noticed that captchas are getting more difficult. Is that because the AI needs to sharpen recognition skills or because that's needed for differentiating human from 'bot?
- sam0x17 3y agobreaking: AI is better than humans at appearing human when completing CAPTCHAs
- robalfonso 3y agoTo that point, when I do the picture captcha (Select the crosswalks type question), I always click a square I know is not valid and then de-select it. Adds some "human-ness" to the interaction and I never get a 2nd challenge that way. Will that be the future? Look for behavior that is too perfect?
- mewpmewp2 3y agoThis would be fairly simple to simulate using an AI though.
- wolpoli 3y agoI feel like Captcha already takes into account of how quickly I select the pictures already. If I spend the time to get it perfect, I end up with more challenges than if I just select quickly based on my instincts.
- wongarsu 3y agoIf you assume that most humans spend the minimum effort possible on their captchas, your gut response is also more in line with other responses than your well thought-out response. Even a matching based on the selected squares would pick up on this.
- whimsicalism 3y agoI do the same but with erratic mouse movements
- RetroTechie 3y agoSo in near future, humans will have to out-do bots in the art of make-human-looking mistakes? Humans will lose!! lol ;-)
- dave1010uk 3y agoSometimes I click the "I am not a robot" checkbox without even thinking. Then I panic for a moment as there's no option to uncheck.
- juujian 3y agoDo CAPTCHAs in practice really serve to perfectly tell apart humans or robots, or just to thwart the laziest attempt of abusing a website? Everyone knows that a door lock, no matter how good, can be overcome if you a really determined. But the lock still thwarts off opportunists. Yes, the goal when we first developed CAPTCHAs may have been to tell apart humans and robots. Realistically now, if CAPTCHAs can still significantly reduce the traffic from bots then companies will keep using them.
- danaris 3y agoThe difference with a door lock is, even if you are sufficiently determined and have a way of defeating particular kinds of door lock with 95% confidence, that doesn't mean that you can instantly break into 95% of houses that use that lock, because you are one person with a physical body. If bots that can break CAPTCHAs become widespread, the volume of spam, scams, and other junk traffic is going to cause problems for many people and small websites.
- stavros 3y agoNo, because bots that handle spam get better as well: https://thespamchronicles.stavros.io/welcome/ https://thespamchronicles.stavros.io/welcome/
- wongarsu 3y agoFor a long time now CAPTCHAs only make bots more expensive. There are plenty of (human-staffed) services that will solve your captchas for $0.001-$0.005 per solved captcha. Better AI lowers the cost, but it's not necessarily a massive change of the status quo.
- RetroTechie 3y agoIn a way that is silly: Increase the cost of having bots do it, while... ...also increasing the time wasted by humans. Time which is considered much more costly/valuable than whatever $ value spent on the bots. So maybe the time has come to regard captcha's pointless, and just drop that nonsense.
- spondylosaurus 3y ago> maybe we should flip the acceptance criteria to make sure you are not performing task at superhuman level Many CAPTCHAs already operate this way. "Bots can do things at suspiciously superhuman speed" isn't new.
- bfeynman 3y agoI said level not speed, obviously something sending solve request back less than a few milliseconds later is not human.
- fnordpiglet 3y agoI have to solve all the captchas for my wife. I don’t know what that says about her, or me, but I definitely loathe them.
- CamperBob2 3y agoWhat it says is that one viable way to get rid of these stupid things may involve ADA lawsuits. Hopefully anti-discrimination lawyers are paying attention. Google has some pretty deep pockets, as do many websites that employ CAPTCHAS. Failing that, we can't be more than a couple of years away from generalized solvers that can simply be implemented as browser plugins, at least on the desktop. The job of coming up with a fair, equitable and non-discriminatory test that only humans can pass is going to be an impossible one.
- Zetice 3y agoWe do this with some "slide this puzzle piece into place" CAPCHAs; my understanding is the detection is based on how slowly you fit the piece in. A computer would be linear in its movement, whereas a human with a mouse would operate with some unevenness and/or slowness and/or inaccuracy.
- jacobr1 3y agoWhich seems easy enough to program around if you are trying to commit fraud against whatever is gated by these tools.
- morkalork 3y agoGet humans to do a few hundred, log their movements, sample from the distribution, sleep(t), and voilà.
- Zetice 3y agoOne would presume it's harder than that, given it's the product of a team/company specializing in detecting exactly such a thing, right?
- brokenmachine 3y agoIf you look behind the curtain in a lot of industries, many of them are held together with chewing gum and string, but have big marketing departments.
- Zetice 3y agoSure, but less so on the core value prop for that company.
- brokenmachine 3y agoI feel like replaying from 100 randomly-selected but real human movements would be pretty hard to detect.
- soerxpso 3y ago> On an unrelated note, I have found captchas that don't even work and reprompt me all the time Modern captchas do a lot of background work regarding how human your inputs look, whether you have a human-seeming fingerprint, etc. Often when I'm behind a VPN and on Linux I have the same issue, because my setup simply looks "too botty" no matter how good I am at telling which squares have a firetruck in them.
- thebruce87m 3y ago> On an unrelated note, I have found captchas that don't even work and reprompt me all the time This is by design. The ones where you have to identify a bus, crosswalk etc are all used to train ML models. Your results are checked against other for the captcha, but sometimes you are the first person to see the image and there’s no way to check your answer so you’ll always get served another. Another smart thing is that they actually segment the picture by moving the squares slightly.
- nickcw 3y ago> This is by design. The ones where you have to identify a bus, crosswalk etc are all used to train ML models. Your results are checked against other for the captcha, but sometimes you are the first person to see the image and there’s no way to check your answer so you’ll always get served another. Do you have a reference for this? I wouldn't have thought a process like that would be needed now-a-days for training ML models.
- thebruce87m 3y agoNot sure if there is anything directly from the horse’s mouth, but there are lots of articles describing it: https://www.techradar.com/news/captcha-if-you-can-how-youve-been-training-ai-for-years-without-realising-it https://www.techradar.com/news/captcha-if-you-can-how-youve-... Human labels are absolutely still needed, for now at least.
- mcast 3y agoGoogle was using CAPTCHA data to train its Street View photos for addresses and street names back in 2012*. https://techcrunch.com/2012/03/29/google-now-using-recaptcha-to-decode-street-view-addresses/amp/ https://techcrunch.com/2012/03/29/google-now-using-recaptcha...
- jacurtis 3y agoSo it is actually a little different than what is noted above. I actually was told this directly from the mouth of someone who worked on this project. I don't believe it is that secret. But this is how it works. The Captcha presents you with 9 squares. It selects a identification test at random (crosswalks, trains, buses, stoplights, etc). For this example let's say the identification test is to identify crosswalks. The squares are then filled as follows: 1) Two of the squares are requested that pass the identification test at an alpha value p < 0.05 (meaning it is more than 95% confident it IS a crosswalk). 2) One square is requested that passes the identification test at an alpha value of p < 0.01 (meaning 99%+ confident, effectively certain it IS a crosswalk) 3) One square is requested that fails the identification test at an alpha value of p < 0.01 (it is almost certainly NOT a crosswalk) 4) Two squares are requested that fail the identification test at an alpha value of p < 0.05 (it is 95% confident that it is NOT a crosswalk) 5) Three squares are requested that need have low confidence intervals p > 0.05 The captcha then shuffles these 9 images at random, it offsets the images a little bit by altering the crop slightly to prevent memorization by bots. Then it presents these 9 squares to the users asking them to identify according to the identification test. The captcha scores the user based on their selection with the 6 known squares. The response you give on the 3 low-confidence squares has zero impact on you passing or failing the test. From what I was told, you must successfully identify both of the 99% interval squares correctly (one that passes the id test and one that doesn't). That is a hard pass/fail. From there, the captcha scores your response on the 95% confidence interval squares to the expected values. It compares that to other variables such as the speed that you answer them, the movement of the cursor and other variables (such as selecting, deselecting, etc). It also compares IP address google session data as part of its determination to determine the liklihood of humanity in the user. My understanding is that is is moderately forgiving. If the user is determined to be human based on those responses, then your responses are fed back into the confidence intervals for all of the images presented (other than the two "known" squares). Data by users that fail the Captcha is discarded so it doesn't feed into the confidence metrics of the images presented. From what I was told, you can actually incorrectly identify 2 squares and still pass the captcha. The IP address and mouse movement plays a significant impact in the response as well as your ability to identify the two known squares. Three of the squares are entirely unknown to the bot. You are purely feeding the confidence on those images for future use in the CAPTCHA and other google products. But there is no test where you are "guaranteed to fail" as mentioned above. Every test presented to you can be passed. There are 2 known squares which you MUST answer correctly. Your behavior and computer data and answers on the mid-confidence squares are what further impact your pass/fail determination. The three unknown squares never impact your pass rate. They are filler, the captcha only watches how you interact with the filler squares, not what you actually respond.
- toss1 3y agoYup. This quote jumped out at me: >>“Furthermore the bots’ solving times are significantly lower in all cases, except reCAPTCHA, where human solving time of 18 seconds is nearly similar to the bots’ time of 17.5 seconds.” This is currently an obvious tell for the standard CAPTCHAs, as you mentioned, "performing at a superhuman level". However, it's an easy bot behavioral fix, so.... what is the next step? Offer a game of chess and look for a human the playing style? Seems you'd have to offer a menu of games, but not "Global Thermo...."
- jonny_eh 3y ago> I have found captchas that don't even work and reprompt me all the time That's the Epic Games Store for me.
- benbristow 3y agowait(random(sensiblerange)); Job done! Even maybe add a answer = getrandombool(somesensiblepercentage) : rightanswer() : wronganswer(); For good measure
- Jackson__ 3y ago>CAPTCHAs are supposed to tell humans and robots apart, Nowadays, it seems the kind of captchas I get when under suspicion of being a bot are simply there to delay. Especially google captcha with their extremely slow fade out box selections.
- Sohcahtoa82 3y ago> On an unrelated note, I have found captchas that don't even work and reprompt me all the time, I wonder if there is some naive filtering behavior they are applying. Curious, do you use a VPN or Tor? Either of those will cause CAPTCHAs to make you solve multiple puzzles. The only site I visit that has ever been annoying with CAPTCHAs is PCPartPicker.
- nyolfen 3y agowe just need to ask it to say a slur
- langcalvin 3y agoThis is an interesting take and I've seen this done for determining if someone is cheating at Chess. If a player's movements match the AI moves too much (85%+ I think) then the player is considered cheating.