3 ms·

Well, you're comparing apples to oranges. You're comparing a hand coded attack against a specific captcha (a guess at the position/vertical lines) to no one mak

by jere 14y ago

Well, you're comparing apples to oranges. You're comparing a hand coded attack against a specific captcha (a guess at the position/vertical lines) to no one making the attempt at all. Attacking your site, with minimal effort, would have yielded a 100% success rate. Attacking this site with a guess would have yielded a 3% rate. Some of the best captchas have been attacked with success rates in the high double digits: http://en.wikipedia.org/wiki/CAPTCHA#Computer_character_recognition http://en.wikipedia.org/wiki/CAPTCHA#Computer_character_reco... So 3% doesn't look too bad after all. I do, by the way, agree with you that the vertical lines attack would work really well.

mikeash 14y ago
I'm a little confused as to why you spend three paragraphs discussing the random guessing approach and only touch on the more sophisticated method at the end. I really don't think I'm comparing apples and oranges. I'm simply noting that any novel captcha can look good when attackers can't be bothered to actually attack it.
- jere 14y ago
  Well, mainly because I agreed with you and didn't feel like wasting much time saying so. I was also frustrated with so many people in this thread focusing on the guessing method and acting like a 3% success rate was high. That annoyed me since, as I said, first class captchas have been attacked with ~50% success rate in studies. My experience with an unknown site has differed from yours dramatically. I have a small, custom site that I share with less than 10 friends and I have had to progressively step up the security (none -> captchas -> passwords) because it has been attacked with thousands of spam messages on several occasions.
  - mikeash 14y ago
    Odd that your experience has been so different. My progression has been: 1. The "elbow" technique. Worked OK for years, then spam began to rise. 2. A slight variation to defeat generalized bots. I changed the field label to, "Type 'humour' with American spelling." This held on for a few more years. 3. A hashcash-based solution, where a proof of work is generated using JavaScript in the client and required to post a comment. It's calibrated to take 20-30 seconds on typical computers, and doesn't start until you focus the comment fields so it's not burning CPU for people who just view the page. This is still working well for me. I get occasional spams, but from looking at the logs it appears almost certain that these are from real humans who are actually spending the 20-30 seconds waiting for the hashcash to compute. I only have to delete one or two spam comments a month for the moment. Wonder what caused your site to suffer so much more.
jwilkins 14y ago
What do you mean by 'some of the best'? If a CAPTCHA can be solved or guessed in an automated fashion then attackers can just throw more (likely compromised) machines at the problem at little cost. 3% is awful. http://bitland.net/captcha.pdf http://bitland.net/captcha.pdf
- jere 14y ago
  In your link, their first attempt at breaking reCaptcha seemed to yield a 17.5% success rate. I was referencing wikipedia, which stated a 60% success rate against Microsoft's captcha and a 30% success rate against Google's catpcha: http://en.wikipedia.org/wiki/Captcha#Computer_character_recognition http://en.wikipedia.org/wiki/Captcha#Computer_character_reco... Those papers may be a few years old and the state of the art may be different. But after an initial look I'm missing the reason that, compared to these captchas, you think that "3% is awful." >If a CAPTCHA can be solved or guessed in an automated fashion then attackers can just throw more (likely compromised) machines at the problem at little cost. I'm not ready to buy this. I would think every captcha is going to have some failure rate, even if it is extremely low. If attacks were absolutely free, then it wouldn't matter what the attack success rate was. Computers are fast, but not infinitely fast. Bandwidth is cheap, but not infinitesimally cheap.
  - jwilkins 14y ago
    CAPTCHA comes from "Completely Automated Public Turing test to tell Computers and Humans Apart". These tests don't fulfill that requirement. Random guessing gives you 3%, which is worse than random guessing on either the MSFT or reCAPTCHA. This is far worse though. A simple loop over the 30 positions, running the output through an OCR engine would give you nearly 100%. FWIW, I wrote the paper and AFAIK was the first person to break reCAPTCHA. I worked on the original MSFT Passport/Hotmail CAPTCHA system and improved MySpace's CAPTCHA which took spammer registrations from ~1,000,000/day (automated) to a few thousand (manual) in late 2007.

by jere 14y ago

mikeash 14y ago

I'm a little confused as to why you spend three paragraphs discussing the random guessing approach and only touch on the more sophisticated method at the end. I really don't think I'm comparing apples and oranges. I'm simply noting that any novel captcha can look good when attackers can't be bothered to actually attack it.

jere 14y ago
Well, mainly because I agreed with you and didn't feel like wasting much time saying so. I was also frustrated with so many people in this thread focusing on the guessing method and acting like a 3% success rate was high. That annoyed me since, as I said, first class captchas have been attacked with ~50% success rate in studies. My experience with an unknown site has differed from yours dramatically. I have a small, custom site that I share with less than 10 friends and I have had to progressively step up the security (none -> captchas -> passwords) because it has been attacked with thousands of spam messages on several occasions.
- mikeash 14y ago
  Odd that your experience has been so different. My progression has been: 1. The "elbow" technique. Worked OK for years, then spam began to rise. 2. A slight variation to defeat generalized bots. I changed the field label to, "Type 'humour' with American spelling." This held on for a few more years. 3. A hashcash-based solution, where a proof of work is generated using JavaScript in the client and required to post a comment. It's calibrated to take 20-30 seconds on typical computers, and doesn't start until you focus the comment fields so it's not burning CPU for people who just view the page. This is still working well for me. I get occasional spams, but from looking at the logs it appears almost certain that these are from real humans who are actually spending the 20-30 seconds waiting for the hashcash to compute. I only have to delete one or two spam comments a month for the moment. Wonder what caused your site to suffer so much more.

jwilkins 14y ago

What do you mean by 'some of the best'? If a CAPTCHA can be solved or guessed in an automated fashion then attackers can just throw more (likely compromised) machines at the problem at little cost. 3% is awful. http://bitland.net/captcha.pdf http://bitland.net/captcha.pdf

jere 14y ago
In your link, their first attempt at breaking reCaptcha seemed to yield a 17.5% success rate. I was referencing wikipedia, which stated a 60% success rate against Microsoft's captcha and a 30% success rate against Google's catpcha: http://en.wikipedia.org/wiki/Captcha#Computer_character_recognition http://en.wikipedia.org/wiki/Captcha#Computer_character_reco... Those papers may be a few years old and the state of the art may be different. But after an initial look I'm missing the reason that, compared to these captchas, you think that "3% is awful." >If a CAPTCHA can be solved or guessed in an automated fashion then attackers can just throw more (likely compromised) machines at the problem at little cost. I'm not ready to buy this. I would think every captcha is going to have some failure rate, even if it is extremely low. If attacks were absolutely free, then it wouldn't matter what the attack success rate was. Computers are fast, but not infinitely fast. Bandwidth is cheap, but not infinitesimally cheap.
- jwilkins 14y ago
  CAPTCHA comes from "Completely Automated Public Turing test to tell Computers and Humans Apart". These tests don't fulfill that requirement. Random guessing gives you 3%, which is worse than random guessing on either the MSFT or reCAPTCHA. This is far worse though. A simple loop over the 30 positions, running the output through an OCR engine would give you nearly 100%. FWIW, I wrote the paper and AFAIK was the first person to break reCAPTCHA. I worked on the original MSFT Passport/Hotmail CAPTCHA system and improved MySpace's CAPTCHA which took spammer registrations from ~1,000,000/day (automated) to a few thousand (manual) in late 2007.