6 ms·

It's actually pretty interesting to see how the captchas have evolved as, presumably, Google decides "OK, we have enough data to consistently identify this thin

It's actually pretty interesting to see how the captchas have evolved as, presumably, Google decides "OK, we have enough data to consistently identify this thing" and moves on to the next challenge. I recall the modern (non-text) captchas used to be cars pretty much every time. Then, the images started getting grainier as they apparently wanted to improve their recognition in different conditions. Then crosswalks and store fronts became quite common, eventually with the same kinds of noise distorting images. Now I've started seeing things like buses, bridges, motorcycles, bicycles, etc. It feels like they've finished getting enough data for improving Google Maps and have begun moving towards collecting data for their self-driving car projects.

cmroanirgo 8y ago
Yep. With the possibility of almost uniquely identifying us on the web through fingerprinting... Google, of all companies is in the perfect position to know that my web request was made by me... And therefore I'm not a robot. You can only conclude that recaptcha is a ml training exercise.
- bpicolo 8y ago
  >You can only conclude that recaptcha is a ml training exercise. They're not secretive about it https://developers.google.com/recaptcha/ https://developers.google.com/recaptcha/
- DigitalBison 8y ago
  > With the possibility of almost uniquely identifying us on the web through fingerprinting... Google, of all companies is in the perfect position to know that my web request was made by me... And therefore I'm not a robot. The article explains that this is part of what reCAPTCHA does, e.g.: > Finally they combine all of this data with their knowledge of the person using the computer. Almost everyone on the Internet uses something owned by Google – search, mail, ads, maps – and as you know Google Tracks All Of Your Things. When you click that checkbox, Google reviews your browser history to see if it looks convincingly human. But your point is otherwise right in that it's used for ML training, which Google admits as another commenter pointed out.
  - gregknicholson 8y ago
    > Finally they combine all of this data with their knowledge of the person using the computer. Almost everyone on the Internet uses something owned by Google – search, mail, ads, maps – and as you know Google Tracks All Of Your Things. Human [n]: Entity that uses Google®-brand services. — Google Dictionary, 2020 edition
- felipemnoa 8y ago
  Maybe, maybe not. From time to time google forces me to prove I'm not a robot because of unusual search queries. Either my queries are unusual or I have an infected computer. The thing is, it also happens to me on my iPhone. Or, maybe they feel I'm not pulling my own weight, seeing as I rarely ever click on adds. They probably need more monkeys to feed the beast so I get selected to train their AI beast. Edit: It could also be that I'm always running on incognito mode.
CharlesColeman 8y ago
> Then, the images started getting grainier as they apparently wanted to improve their recognition in different conditions. I'd always assumed that was noise carefully tuned to throw off one machine learning model or another that was being used to beat the captcha, sort of like this: https://www.theverge.com/2017/11/2/16597276/google-ai-image-attacks-adversarial-turtle-rifle-3d-printed https://www.theverge.com/2017/11/2/16597276/google-ai-image-... I think it might be the same when they switch to other types of objects (like crosswalks or bikes). Someone's model got too good, so they had to change to something else. I also get the impression that they add delays to the tile refresh before they do that. I suspect Google now uses robots to generate captchas for humans, under the assumption their image recognizers are far better than anyone else's. They already have some very well ones for other products (self driving cars, street view) and lots of street-level city imagery. That would explain why their captchas are so difficult for humans to solve -- they're testing if you see things like their "AI," not like other humans.
- dylan604 8y ago
  >I'd always assumed that was noise carefully tuned to throw off one machine learning model or another that was being used to beat the captcha I was thinking it was trying to dirty up the image just like the lenses on cameras get dirty. What happens to the image recognition when there's water spots, dirt, mud, etc on the lens that keeps parts of the image obscured?
  - scrollaway 8y ago
    The images get a lot grainier when you browse over VPN or fail the first round. So I suspect it's meant to actually throw off captcha-solvers.
  - jameshart 8y ago
    what would be the training data value in getting humans to classify images you have clean versions of, after applying simulated noise? If you have the clean version of the image, you need to get that classified by a human - then you can throw noisy versions of it into the training set for your AI. You don’t need to ask a human, hey, I added noise to a picture of a yield sign. Is it still a picture of a yield sign?
taneq 8y ago
Google: spins off robotic car company Waymo, announcing public-facing robotaxi service launch in 2018 Also Google: "Our standard for 'what a machine couldn't possibly do' is identifying a stop sign."
- UncleMeat 8y ago
  Machines have been able to beat captchas for years and years. The point is that it costs money and that's good enough to prevent free and scalable abuse.
- TrickyRick 8y ago
  Isn't the captcha data used specifically to train that AI? That's what they did with the old one at least where it was used to train machines to read.
- deleted 8y ago
  [deleted]
pbhjpbhj 8y ago
Those really grainy images, they always seemed like they had to have noise added on purpose? It was like really badly processed film grain, but if the images were enlargements then wouldn't they be pixelated? Stuff you get now often requires cultural information, like "sidewalk" isn't a cross-cultural name, I'd guess almost everyone knows it, but meh. What classes as a store, is a lawyers office a store? Also, I seem to recall I had "click on all minivans"?? Not sure what one of those is, nor really what is classed as a car in USA, is an MPV a car [I'd guess that's what a minivan is?]? Do pedestrian crossing lights (green/red man) count as [part of] traffic lights? I've often wanted a short description of the locus of the terms they're using. Of course it never tells you if you failed, just gives you a further captcha, which it might have done anyway.
- kalleboo 8y ago
  The fire hydrant one always annoyed me - in many countries stand-up fire hydrants are nonexistent or at least look very different. I guess they're banking on people having seen enough American movies/TV shows?
  - kaybe 8y ago
    It's not like the word 'fire hydrant' exists in those countries either, so some kind of knowledge/learning/look-up is involved anyway, no?
    - tomatocracy 8y ago
      In the UK we call them fire hydrants and they’re typically placed underground with a small metal cover in the middle of the road and a sign on the pavement (sidewalk for Americans) saying where it is.
    - deleted 8y ago
      [deleted]
- dzhiurgis 8y ago
  Remember this belongs to Google https://patents.google.com/patent/US9407661B2/en https://patents.google.com/patent/US9407661B2/en
tushartyagi 8y ago
And I find it equally interesting to think about what happened to Google Books project after half the world was entering text into captchas perhaps close to a decade? Is the project still going on?
seventhtiger 8y ago
Eventually you will have to park a self driving car in a tight spot for captcha.
arayh 8y ago
I wouldn't be surprised if soon enough you'll have to play a mini-match of Starcraft 2 as part of REcaptcha to train DeepMind.

cmroanirgo 8y ago

Yep. With the possibility of almost uniquely identifying us on the web through fingerprinting... Google, of all companies is in the perfect position to know that my web request was made by me... And therefore I'm not a robot. You can only conclude that recaptcha is a ml training exercise.

bpicolo 8y ago
>You can only conclude that recaptcha is a ml training exercise. They're not secretive about it https://developers.google.com/recaptcha/ https://developers.google.com/recaptcha/
DigitalBison 8y ago
> With the possibility of almost uniquely identifying us on the web through fingerprinting... Google, of all companies is in the perfect position to know that my web request was made by me... And therefore I'm not a robot. The article explains that this is part of what reCAPTCHA does, e.g.: > Finally they combine all of this data with their knowledge of the person using the computer. Almost everyone on the Internet uses something owned by Google – search, mail, ads, maps – and as you know Google Tracks All Of Your Things. When you click that checkbox, Google reviews your browser history to see if it looks convincingly human. But your point is otherwise right in that it's used for ML training, which Google admits as another commenter pointed out.
- gregknicholson 8y ago
  > Finally they combine all of this data with their knowledge of the person using the computer. Almost everyone on the Internet uses something owned by Google – search, mail, ads, maps – and as you know Google Tracks All Of Your Things. Human [n]: Entity that uses Google®-brand services. — Google Dictionary, 2020 edition
felipemnoa 8y ago
Maybe, maybe not. From time to time google forces me to prove I'm not a robot because of unusual search queries. Either my queries are unusual or I have an infected computer. The thing is, it also happens to me on my iPhone. Or, maybe they feel I'm not pulling my own weight, seeing as I rarely ever click on adds. They probably need more monkeys to feed the beast so I get selected to train their AI beast. Edit: It could also be that I'm always running on incognito mode.

CharlesColeman 8y ago

> Then, the images started getting grainier as they apparently wanted to improve their recognition in different conditions. I'd always assumed that was noise carefully tuned to throw off one machine learning model or another that was being used to beat the captcha, sort of like this: https://www.theverge.com/2017/11/2/16597276/google-ai-image-attacks-adversarial-turtle-rifle-3d-printed https://www.theverge.com/2017/11/2/16597276/google-ai-image-... I think it might be the same when they switch to other types of objects (like crosswalks or bikes). Someone's model got too good, so they had to change to something else. I also get the impression that they add delays to the tile refresh before they do that. I suspect Google now uses robots to generate captchas for humans, under the assumption their image recognizers are far better than anyone else's. They already have some very well ones for other products (self driving cars, street view) and lots of street-level city imagery. That would explain why their captchas are so difficult for humans to solve -- they're testing if you see things like their "AI," not like other humans.

dylan604 8y ago
>I'd always assumed that was noise carefully tuned to throw off one machine learning model or another that was being used to beat the captcha I was thinking it was trying to dirty up the image just like the lenses on cameras get dirty. What happens to the image recognition when there's water spots, dirt, mud, etc on the lens that keeps parts of the image obscured?
- scrollaway 8y ago
  The images get a lot grainier when you browse over VPN or fail the first round. So I suspect it's meant to actually throw off captcha-solvers.
- jameshart 8y ago
  what would be the training data value in getting humans to classify images you have clean versions of, after applying simulated noise? If you have the clean version of the image, you need to get that classified by a human - then you can throw noisy versions of it into the training set for your AI. You don’t need to ask a human, hey, I added noise to a picture of a yield sign. Is it still a picture of a yield sign?

taneq 8y ago

Google: spins off robotic car company Waymo, announcing public-facing robotaxi service launch in 2018 Also Google: "Our standard for 'what a machine couldn't possibly do' is identifying a stop sign."

UncleMeat 8y ago
Machines have been able to beat captchas for years and years. The point is that it costs money and that's good enough to prevent free and scalable abuse.
TrickyRick 8y ago
Isn't the captcha data used specifically to train that AI? That's what they did with the old one at least where it was used to train machines to read.
deleted 8y ago
[deleted]

pbhjpbhj 8y ago

Those really grainy images, they always seemed like they had to have noise added on purpose? It was like really badly processed film grain, but if the images were enlargements then wouldn't they be pixelated? Stuff you get now often requires cultural information, like "sidewalk" isn't a cross-cultural name, I'd guess almost everyone knows it, but meh. What classes as a store, is a lawyers office a store? Also, I seem to recall I had "click on all minivans"?? Not sure what one of those is, nor really what is classed as a car in USA, is an MPV a car [I'd guess that's what a minivan is?]? Do pedestrian crossing lights (green/red man) count as [part of] traffic lights? I've often wanted a short description of the locus of the terms they're using. Of course it never tells you if you failed, just gives you a further captcha, which it might have done anyway.

kalleboo 8y ago
The fire hydrant one always annoyed me - in many countries stand-up fire hydrants are nonexistent or at least look very different. I guess they're banking on people having seen enough American movies/TV shows?
- kaybe 8y ago
  It's not like the word 'fire hydrant' exists in those countries either, so some kind of knowledge/learning/look-up is involved anyway, no?
  - tomatocracy 8y ago
    In the UK we call them fire hydrants and they’re typically placed underground with a small metal cover in the middle of the road and a sign on the pavement (sidewalk for Americans) saying where it is.
  - deleted 8y ago
    [deleted]
dzhiurgis 8y ago
Remember this belongs to Google https://patents.google.com/patent/US9407661B2/en https://patents.google.com/patent/US9407661B2/en

tushartyagi 8y ago

And I find it equally interesting to think about what happened to Google Books project after half the world was entering text into captchas perhaps close to a decade? Is the project still going on?

seventhtiger 8y ago

Eventually you will have to park a self driving car in a tight spot for captcha.

arayh 8y ago

I wouldn't be surprised if soon enough you'll have to play a mini-match of Starcraft 2 as part of REcaptcha to train DeepMind.