7 ms·

Google CAPTCHAs were designed and deployed as a mechanism to train AIs. That's why they are the way they are. Any security theater surrounding them is entirely

Google CAPTCHAs were designed and deployed as a mechanism to train AIs. That's why they are the way they are. Any security theater surrounding them is entirely incidental. So it's no surprise that the AIs are now good at solving them. We've trained them for years.

leobg 3y ago
I still find it funny that Google, with the advantage of having millions of Internet users train their AI like galley slaves for free, hasn’t yet been able to crack vision driven self driving. Tesla had no such advantage when training their FSD to recognize traffic lights, bicycles, motorcycles, etc.
- mike_d 3y ago
  > hasn’t yet been able to crack vision driven self driving But they have? For years Google Street view has read signs, house numbers, phone numbers of businesses, etc. from the environment. It is safe to assume they have this built into Waymo as well. I assume you might be trying to reference "vision only" self-driving, which is a fantasy made up by Elon Musk because nobody would sell him LiDAR sensors cheaply. https://www.thedrive.com/tech/43779/this-tesla-model-y-dummy-crash-shows-exactly-why-lidar-matters https://www.thedrive.com/tech/43779/this-tesla-model-y-dummy...
  - leobg 3y ago
    This is a meme. “Sour grape Elon, touting vision because no one will sell him LiDAR sensors. Which are the gold standard sensors that solve self driving.” How exactly does LiDAR tell you whether the thing in question can move (dog) or not (trash can)? How does it allow a neural net to infer intent? You’ll actually have to solve vision. Even if you had LiDAR. There’s no way around it. And once you’ve solved it, LiDAR becomes superfluous. Chesterton’s fence.
    - mike_d 3y ago
      > How exactly does LiDAR tell you whether the thing in question can move LIDAR is continuously scanning, usually multiple times a second. It is irrelevant if the object is a trash can or a dog if it has a trajectory into the street. > And once you’ve solved it, LiDAR becomes superfluous. Only if you have the low Musk level standards of simply being equal to a human. There are plenty of jobs robots can do better than humans and driving is one of them. But it does require LIDAR and/or radar. https://abc7news.com/tesla-s-autopilot-self-driving-car-officials-investigating-teslas-autopilot-feature-after-fatal-crash/1410042/ https://abc7news.com/tesla-s-autopilot-self-driving-car-offi...
- eviks 3y ago
  It's a much harder problem, and Tesla is nowhere close to the solution
- noduerme 3y ago
  Tesla, the company that just recalled 2 million self driving cars? In fairness, the company best positioned to harness user input to an AI that avoids crashes would probably be Rockstar. OTOH, that AI would definitely not obey stop signs or pedestrians.
  - hehhehaha 3y ago
    "recall"
  - bheadmaster 3y ago
    Tesla recalled two million vehicles after federal officials said it had not done enough to make sure that drivers remained attentive when using the system. Not because their self-driving system sucks, or whatever you were trying to imply.
    - diputsmonro 3y ago
      If the self driving system were worth it's salt, it wouldn't matter if the drivers weren't paying attention. Ergo, the system sucks, or is at the very least not nearly as good as Tesla likes to tout.
      - rezonant 3y ago
        Well it's not like there's a self driving car system in operation today that does not require a human in the driver seat at all. Waymo has so much catching up to do.
        deleted 3y ago
        [deleted]
      - FergusArgyll 3y ago
        It can be "worth it's salt" but the government still doesn't see it as such (for many possible reasons). I don't know if it is or isn't, I never drove one, but those are two completely different standards
- NotSammyHagar 3y ago
  The tesla system is exciting and dangerous, because it does identify many things in the environment, but it's extremely unsafe because on city driving it will not make the right choice most of the time. On the freeway it does much better, but then that's a more restricted environment. I have an older tesla S with the pre-ai so called autopilot. It has one camera in the front and a radar and the system detects a few things like speed limit signs. The main extent of what it can do is follow the current lane pretty wall, even when it curves, slows down if it comes up to a car going slower than its preset speed. The good thing is it works on any road. It does a shockingly good job. The later systems with onboard special processors are like a crazy beginning driver to has way too much confidence and drives in dangerous situations willy nilly. There are many other people who have explored it and written long posts. It's not safe. You can try to use it be you have to be constantly paying extreme attention. It's like watching your kid drive the first time. I know you should be watching the stupid ai all the time, but it's far from being safe.
  - pixl97 3y ago
    Yea, that's the problem with self driving, especially in cities/dense areas. We really need AGI first. There are so many issues that humans react to before there is identifiable danger. "Good" drivers see questionable situations and slow down or position themselves farther from potential issues before they get to the issue so they don't have to react at the last minute.
noduerme 3y ago
All true, except: While these are considered just an excruciating security pain for users, they do serve a non-theatrical purpose in many cases of throttling the speed of brute force attacks (or at least costing your opponent money).
- snordgren 3y ago
  GPT-4 (in)famously tricked a human to do a captcha for it. The current GPT-4 with vision would probably have been able to do it without the human, but maybe it has been “gaslit” by all the content online saying that only humans can solve captchas, that it doesn’t consider it?
  - stavros 3y ago
    I really doubt that GPT-4 had the "will" to do anything. Someone must have asked it to "want" to trick a user.
    - JimDabell 3y ago
      It’s from here: https://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf (search for "CAPTCHA"). It was an artificial exercise that got massively exaggerated. It was explicitly instructed to do nefarious things like lie to people, it didn’t do those things of its own accord.
      - stavros 3y ago
        Thank you for the link, I had found it after some Googling but neglected to post. Yep, they instructed GPT-4 to be nefarious, and it followed the instruction. Hardly the AI uprising, though definitely a good tool for anyone, good or evil.
      - IIAOPSW 3y ago
        When I ask it to lie to me, it says its sorry but as an online AI language model it would be unethical...but when I ask it to tell me a story its happy to comply.
panny 3y ago
>So it's no surprise that the AIs are now good at solving them Funnily enough, AI may be better at solving them than people. I've encountered many Google captchas which reject the correct answers, because you know... bots trained it to accept incorrect ones. Anyway, at least it's not stop signs anymore. It must have been truly embarrassing that Google was simultaneously selling "self driving" cars but at the same time demonstrating that stop sign recognition couldn't be done by robots.
- bluGill 3y ago
  When I get those I make it a point to look for borderline areas and try to guess how I could mess with their data.
pyeri 3y ago
Once they get fully trained then how will websites ever distinguish between an intelligent bot and real human? At least now, they are outsourcing that filtering to services like cloudflare. But with this kind of training, how will even cloudflare distinguish between bot and the human?
- jacquesm 3y ago
  The human will be the slower one.
  - candiodari 3y ago
    Yeah, no offence, but sleep(2 + random.sample(coffee + toilet + sneezing + normal response time)) has been a required part of web scrapers since forever. With coffee N(1,5 minutes, 20 seconds), toilet N(4 minutes, 30 seconds), ...
    - rezonant 3y ago
      I guess it depends on how you're scraping. For general web crawling, simply implementing a response time based crawl back off per origin and identifying yourself appropriately in User Agent goes a long way. If you are instead automating Facebook's SPA to pull comments for analysis, then yeah you need to emulate a human, because that's not how they intend you to do it.
    - jacquesm 3y ago
      That's incredibly clever!
- candiodari 3y ago
  The thing about CAPTCHAs is that convnets were already better than the average human at reading most/all visual captchas, since ~2000. You still needed to program the logic of the captcha (it couldn't follow instructions like "find the red lights", but it could take a picture and find the red lights). I wonder when we'll get to the point that employers can't tell the difference between transformers and real humans anymore ...
rezonant 3y ago
This doesn't make sense. reCAPTCHA certainly does what it says on the tin. But the way it does it has almost nothing to do with the challenge the human sees. It's all behavioral analytics, including leveraging Google's collected data to determine how likely a user is a bot before they even load the page. I'm not denying reCAPTCHA is a source of training data for Google -- surely there's no particular reason that every single reCAPTCHA V2 challenge is about identifying traffic objects, and it's not like Google is building a self-driving AI or anything. But that's the business model, not the core feature. And, that training data isn't just given to the developers of captcha solving bots.
- black_puppydog 3y ago
  > including leveraging Google's collected data to determine how likely a user is a bot before they even load the page. And also completely incidentally making the web browsing experience a wee bit less pleasant for people who refuse to have google track their every click. Like users of non-chrome browsers, adblockers etc. Totally incidental I'm sure.
wouldbecouldbe 3y ago
I always thought they used more timing & mouse movement instead of correct answer to verify if your a human.
- Solvency 3y ago
  All of which an AI bot agent can trivially fake.
  - wouldbecouldbe 3y ago
    Hmmm not super easy, unless you now how / what they are checking.
- TrackerFF 3y ago
  So instead of running some script checkbox = getPos(checkbox='notRobot') button = getPos(button='submit') cursor() .transition(pos=checkbox) .click() .transition(pos=button) .click() They now checkbox = getPos(checkbox='notRobot') button = getPos(button='submit') cursor() .sleep(time=random(distribution='human_captcha')) .transition(pos=checkbox , method='human_captcha') .sleep(time=random(distribution='human_captcha')) .click() .sleep(time=random(distribution='human_captcha')) .transition(pos=button, method='human_captcha') .sleep(time=random(distribution='human_captcha')) .click() Where sleep and transitioning are sampled from some random distribution that is close to actual human behavior, which should be pretty trivial to model.
  - wouldbecouldbe 3y ago
    only if you know how

leobg 3y ago

I still find it funny that Google, with the advantage of having millions of Internet users train their AI like galley slaves for free, hasn’t yet been able to crack vision driven self driving. Tesla had no such advantage when training their FSD to recognize traffic lights, bicycles, motorcycles, etc.

mike_d 3y ago
> hasn’t yet been able to crack vision driven self driving But they have? For years Google Street view has read signs, house numbers, phone numbers of businesses, etc. from the environment. It is safe to assume they have this built into Waymo as well. I assume you might be trying to reference "vision only" self-driving, which is a fantasy made up by Elon Musk because nobody would sell him LiDAR sensors cheaply. https://www.thedrive.com/tech/43779/this-tesla-model-y-dummy-crash-shows-exactly-why-lidar-matters https://www.thedrive.com/tech/43779/this-tesla-model-y-dummy...
- leobg 3y ago
  This is a meme. “Sour grape Elon, touting vision because no one will sell him LiDAR sensors. Which are the gold standard sensors that solve self driving.” How exactly does LiDAR tell you whether the thing in question can move (dog) or not (trash can)? How does it allow a neural net to infer intent? You’ll actually have to solve vision. Even if you had LiDAR. There’s no way around it. And once you’ve solved it, LiDAR becomes superfluous. Chesterton’s fence.
  - mike_d 3y ago
    > How exactly does LiDAR tell you whether the thing in question can move LIDAR is continuously scanning, usually multiple times a second. It is irrelevant if the object is a trash can or a dog if it has a trajectory into the street. > And once you’ve solved it, LiDAR becomes superfluous. Only if you have the low Musk level standards of simply being equal to a human. There are plenty of jobs robots can do better than humans and driving is one of them. But it does require LIDAR and/or radar. https://abc7news.com/tesla-s-autopilot-self-driving-car-officials-investigating-teslas-autopilot-feature-after-fatal-crash/1410042/ https://abc7news.com/tesla-s-autopilot-self-driving-car-offi...
eviks 3y ago
It's a much harder problem, and Tesla is nowhere close to the solution
noduerme 3y ago
Tesla, the company that just recalled 2 million self driving cars? In fairness, the company best positioned to harness user input to an AI that avoids crashes would probably be Rockstar. OTOH, that AI would definitely not obey stop signs or pedestrians.
- hehhehaha 3y ago
  "recall"
- bheadmaster 3y ago
  Tesla recalled two million vehicles after federal officials said it had not done enough to make sure that drivers remained attentive when using the system. Not because their self-driving system sucks, or whatever you were trying to imply.
  - diputsmonro 3y ago
    If the self driving system were worth it's salt, it wouldn't matter if the drivers weren't paying attention. Ergo, the system sucks, or is at the very least not nearly as good as Tesla likes to tout.
    - rezonant 3y ago
      Well it's not like there's a self driving car system in operation today that does not require a human in the driver seat at all. Waymo has so much catching up to do.
      - deleted 3y ago
        [deleted]
    - FergusArgyll 3y ago
      It can be "worth it's salt" but the government still doesn't see it as such (for many possible reasons). I don't know if it is or isn't, I never drove one, but those are two completely different standards
NotSammyHagar 3y ago
The tesla system is exciting and dangerous, because it does identify many things in the environment, but it's extremely unsafe because on city driving it will not make the right choice most of the time. On the freeway it does much better, but then that's a more restricted environment. I have an older tesla S with the pre-ai so called autopilot. It has one camera in the front and a radar and the system detects a few things like speed limit signs. The main extent of what it can do is follow the current lane pretty wall, even when it curves, slows down if it comes up to a car going slower than its preset speed. The good thing is it works on any road. It does a shockingly good job. The later systems with onboard special processors are like a crazy beginning driver to has way too much confidence and drives in dangerous situations willy nilly. There are many other people who have explored it and written long posts. It's not safe. You can try to use it be you have to be constantly paying extreme attention. It's like watching your kid drive the first time. I know you should be watching the stupid ai all the time, but it's far from being safe.
- pixl97 3y ago
  Yea, that's the problem with self driving, especially in cities/dense areas. We really need AGI first. There are so many issues that humans react to before there is identifiable danger. "Good" drivers see questionable situations and slow down or position themselves farther from potential issues before they get to the issue so they don't have to react at the last minute.

noduerme 3y ago

All true, except: While these are considered just an excruciating security pain for users, they do serve a non-theatrical purpose in many cases of throttling the speed of brute force attacks (or at least costing your opponent money).

snordgren 3y ago
GPT-4 (in)famously tricked a human to do a captcha for it. The current GPT-4 with vision would probably have been able to do it without the human, but maybe it has been “gaslit” by all the content online saying that only humans can solve captchas, that it doesn’t consider it?
- stavros 3y ago
  I really doubt that GPT-4 had the "will" to do anything. Someone must have asked it to "want" to trick a user.
  - JimDabell 3y ago
    It’s from here: https://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf (search for "CAPTCHA"). It was an artificial exercise that got massively exaggerated. It was explicitly instructed to do nefarious things like lie to people, it didn’t do those things of its own accord.
    - stavros 3y ago
      Thank you for the link, I had found it after some Googling but neglected to post. Yep, they instructed GPT-4 to be nefarious, and it followed the instruction. Hardly the AI uprising, though definitely a good tool for anyone, good or evil.
    - IIAOPSW 3y ago
      When I ask it to lie to me, it says its sorry but as an online AI language model it would be unethical...but when I ask it to tell me a story its happy to comply.

panny 3y ago

>So it's no surprise that the AIs are now good at solving them Funnily enough, AI may be better at solving them than people. I've encountered many Google captchas which reject the correct answers, because you know... bots trained it to accept incorrect ones. Anyway, at least it's not stop signs anymore. It must have been truly embarrassing that Google was simultaneously selling "self driving" cars but at the same time demonstrating that stop sign recognition couldn't be done by robots.

bluGill 3y ago
When I get those I make it a point to look for borderline areas and try to guess how I could mess with their data.

pyeri 3y ago

Once they get fully trained then how will websites ever distinguish between an intelligent bot and real human? At least now, they are outsourcing that filtering to services like cloudflare. But with this kind of training, how will even cloudflare distinguish between bot and the human?

jacquesm 3y ago
The human will be the slower one.
- candiodari 3y ago
  Yeah, no offence, but sleep(2 + random.sample(coffee + toilet + sneezing + normal response time)) has been a required part of web scrapers since forever. With coffee N(1,5 minutes, 20 seconds), toilet N(4 minutes, 30 seconds), ...
  - rezonant 3y ago
    I guess it depends on how you're scraping. For general web crawling, simply implementing a response time based crawl back off per origin and identifying yourself appropriately in User Agent goes a long way. If you are instead automating Facebook's SPA to pull comments for analysis, then yeah you need to emulate a human, because that's not how they intend you to do it.
  - jacquesm 3y ago
    That's incredibly clever!
candiodari 3y ago
The thing about CAPTCHAs is that convnets were already better than the average human at reading most/all visual captchas, since ~2000. You still needed to program the logic of the captcha (it couldn't follow instructions like "find the red lights", but it could take a picture and find the red lights). I wonder when we'll get to the point that employers can't tell the difference between transformers and real humans anymore ...

rezonant 3y ago

This doesn't make sense. reCAPTCHA certainly does what it says on the tin. But the way it does it has almost nothing to do with the challenge the human sees. It's all behavioral analytics, including leveraging Google's collected data to determine how likely a user is a bot before they even load the page. I'm not denying reCAPTCHA is a source of training data for Google -- surely there's no particular reason that every single reCAPTCHA V2 challenge is about identifying traffic objects, and it's not like Google is building a self-driving AI or anything. But that's the business model, not the core feature. And, that training data isn't just given to the developers of captcha solving bots.

black_puppydog 3y ago
> including leveraging Google's collected data to determine how likely a user is a bot before they even load the page. And also completely incidentally making the web browsing experience a wee bit less pleasant for people who refuse to have google track their every click. Like users of non-chrome browsers, adblockers etc. Totally incidental I'm sure.

wouldbecouldbe 3y ago

I always thought they used more timing & mouse movement instead of correct answer to verify if your a human.

Solvency 3y ago
All of which an AI bot agent can trivially fake.
- wouldbecouldbe 3y ago
  Hmmm not super easy, unless you now how / what they are checking.
TrackerFF 3y ago
So instead of running some script checkbox = getPos(checkbox='notRobot') button = getPos(button='submit') cursor() .transition(pos=checkbox) .click() .transition(pos=button) .click() They now checkbox = getPos(checkbox='notRobot') button = getPos(button='submit') cursor() .sleep(time=random(distribution='human_captcha')) .transition(pos=checkbox , method='human_captcha') .sleep(time=random(distribution='human_captcha')) .click() .sleep(time=random(distribution='human_captcha')) .transition(pos=button, method='human_captcha') .sleep(time=random(distribution='human_captcha')) .click() Where sleep and transitioning are sampled from some random distribution that is close to actual human behavior, which should be pretty trivial to model.
- wouldbecouldbe 3y ago
  only if you know how