17 ms·

Ask HN: A Good Alternative for ReCaptcha?

368 points by Ayesh 7y ago

Are there any good alternatives for reCaptcha? It has come to a point that traditional text/sound captcha challenges are trivial to bypass today, and the more distorted we make them, the more challenging they are for humans. Google went Sparta with their reCaptcha be, and nobody in their right mind should add a script that fingerprints users, specially from an adtech company. What solutions do you use to thwart bad boots from submitting your forms and automating things where it should not have been?

StreamBright 7y ago
Just out of curiosity, isn't that feasible today to implement some machine learning to stop spammers? Is there any project trying to come from this angle?
- lucb1e 7y ago
  That's like saying "why don't you use algoritms and code". Like, sure, but what is it you're proposing? What features would you learn from and match against? (For those unfamiliar with algoritms and code as solution, it's a reference to this: https://www.reddit.com/r/ProgrammerHumor/comments/5ylndv/so_thats_how_they_did_it_its_brilliant/ https://www.reddit.com/r/ProgrammerHumor/comments/5ylndv/so_... )
  - StreamBright 7y ago
    Actually we have implemented something like that for HTTP requests. Features would be: IP (first 3 octets are probably enough), posting time, length, time to solve captcha, time between clicks, country where the IP is located, post contains certain words (can be learnt from spam posts), does the post contain a link(y/n) I think I would start with these, probably looking into what other people are doing.
    - Ayesh 7y ago
      An ideal machine learning implementation would also need the context, such as the original post itself, parent comment(s), other comments in the thread, etc. It can be quite difficult than one might think. For example, now that we are talking about spam, the word "Viagra" shouldn't block my comment, even though my parent post doesn't mention the word or in a situation where nobody else mentioned it.
- Ayesh 7y ago
  Akismet has been doing this for some time I believe. The downside is that you have to send the form submission and other meta data (IP address, etc) to them. This can be better than the current reCaptcha v3 though.
  - StreamBright 7y ago
    Good to know! I am going to check this out.
- kqr 7y ago
  Sure. I know a developer at Distil Networks[1] and that's exactly their core business idea. Apparently it's very hard, though, because they have positioned themselves as an alternative for very high-value targets (like airline tickets) and charge accordingly for their services. It also seems to require some amount of human intervention still. [1]: https://www.distilnetworks.com/ https://www.distilnetworks.com/
  - StreamBright 7y ago
    Good to know, I guess it boils down what sort of features you can use, which might be hard to find. Based on my experience feature hunting can be a lengthy process.
Nextgrid 7y ago
> It has come to a point that traditional text/sound captcha challenges are trivial to bypass today I have yet to see a general-purpose tool to which you can throw any text captcha and it’ll solve it. Just because there are academic papers that demonstrated it once doesn’t mean there’s still a huge barrier to entry in implementing this solution (which spammers won’t do as long as it’s easier to move onto another target). There are paid captcha-solving services out there and even those are still powered by humans even though it’s in their commercial interests to automate the process. Them not doing so further suggests that AI is not there yet.
- dewey 7y ago
  https://github.com/dessant/buster https://github.com/dessant/buster What about this one?
  - Nextgrid 7y ago
    Does it actually work? Speech recognition is often terrible enough in normal conditions, so I don't expect it working well on audio captchas which are often designed to counteract speech recognition.
    - westondeboer 7y ago
      Buster works great! I use it all the time.
Risse 7y ago
In my personal blog I am using "Riddler" Drupal module, and have had good experience: https://www.drupal.org/project/riddler https://www.drupal.org/project/riddler You can create your own Captcha questions / answers. I feel like this is the preferred way of handling spam posts, creating your own custom Captcha implementation.
- Dolores12 7y ago
  Eventually your questions will be answered by humans and added to database.
  - mattigames 7y ago
    Might be good idea to extend the plug-in so adding questions is as easy as sending an SMS, then you can spend less than a minute daily to add a question/answer combo: Color of the sky at night? Black
    - julianlam 7y ago
      #000000
  - barking 7y ago
    If it takes a while, it wouldn't be too painful to update to a new question periodically, i suppose.
- CM30 7y ago
  This is what I do on my own sites too. Of course, given some bots relay the captchas to humans, this arguably works best for a site with a specific niche in mind, since they can ask a significantly more difficult question that only someone interested in the topic would know. Like say, how a PhD maths related community or blog might ask a university level maths question, a chess forum may ask a chess related one and one about a certain gaming franchise might ask questions about said franchise. Bonus points for keeping out people with no genuine interest in the topic.
temp99990 7y ago
I’m a fan of the Chinese-style captchas where you just move a puzzle piece with a slider. I have no idea how defeatable it is vs reCaptcha but it’s far far less painful.
- mpol 7y ago
  What happens here in the background? I can imagine at the frontend you have some JavaScript, where an input field gets filled in or something. There has to be some server side checking as well, otherwise a simple HTTP POST would submit fine. I do like the idea, but if you need JavaScript anyway, why not have some invisible inputs. They work for now.
- manojlds 7y ago
  Yeah, it's very widely used in the crypto world :)
- martin__ 7y ago
  It seems like it is effective. We were getting a lot of automated requests, and right when we put the waterwall on our page, it did a good job of picking out those users and not impacting others. After a few days though, those users were able to start getting through again, but based on the timing between requests, it looks like they might have had to start operating the page manually.
- tgsovlerkhgsel 7y ago
  How do these work? I assume they try to do some machine learning or other magic on the sliding action?
- lucb1e 7y ago
  I don't know these, do you have an example?
  - ivarojha 7y ago
    Geetest is one of them, I've seen it on crypto projects generally, binance for example.
anonymous5133 7y ago
I had a serious problem with bots spamming my forum. I implemented all the usual captchas but none of it worked. What I found interesting was that I was able to defeat the bots simply by "tricking" them. I kept the old forum up but basically set this forum to auto delete the content on it. I then setup a brand new forum and for whatever reason bots don't spam it, at all. It is almost like the bot goes to the original forum, spams it, and then moves on thinking it has completed its mission. I even flat out disabled the captcha to see if anything changed and nothing did. The new forum never got spammed. I have no idea why that happened but it strangely did work. When I do get spam, I don't think that spam is from bots. It is from humans posting instead but that is at least manageable to clean up. It kinda leads me to conclude that each developer has to create "out of the box" solutions instead of some plug and play solution. If a plug and play solution is developed then all the spam bot creators start figuring out ways to crack or simply create a service for human based cracking. If unconventional methods are used on each site then it gets more complicated for the spammers.
brylie 7y ago
It might be worth considering a honeypot approach. E.g. having a field in the form that isn't visible for users that, when filled in, indicates that it is likely a SPAM submission. https://www.projecthoneypot.org/ https://www.projecthoneypot.org/
- gjs278 7y ago
  I do this but I also put a javascript field that is already checked and js unchecks it. not great for no script users but pretty good for any random bot that is using curl or some other scripting language and doesn’t check your trap box.
  - turshija 7y ago
    You can make that checkbox visible by default and put label "I'm not a bot" and uncheck + hide it using JS, that way noscript users will still see the checkbox and uncheck it manually.
    - tylerhou 7y ago
      Bots might be smart enough to uncheck that from the label; maybe text near the top of the form that says “please uncheck the checkbox near the submit button”?
      - wongarsu 7y ago
        You could use CSS to place label and checkbox visually close but completely unrelated in the DOM. Not great for accessibility, but better than the current situation.
mpol 7y ago
Akismet is a third party service that works really well. You send data there with a HTTP POST and it will reply with a yes or no, it is spam or not spam. It is not that hard to implement. You do have to be aware that you are sending user data to that service, which you have to mention in your privacy policy. Stop Forum Spam is a similar third party service. You send it an ip address and an email address. It will reply on both items if it is spam, together with a confidence level. Quite interesting way to reply :) It is originally intended to fight registration spam, but you can use it for comment spam or contact forms as well. JavaScript spamfilters can be very usefull. Most spambots do a HTTP GET for a page with a form. They fill in all the fields and submit it with a HTTP POST. They don't run any JavaScript on that page. You can have honeypot and timeout fields on a form that get manipulated by JavaScript, and spambots will not validate. Works really well, and all transparent to the user. The only "risk" is that in the future spammers might start using more sophisticated spambots, like using Electron or Chromium. I implemented spamfilters like this in a WordPress plugin and it works really well for me: https://wordpress.org/plugins/la-sentinelle-antispam/ https://wordpress.org/plugins/la-sentinelle-antispam/
- avian 7y ago
  > The only "risk" is that in the future spammers might start using more sophisticated spambots You’re also making your website unusable for people with Javascript blocked or disabled in their browsers.
  - mpol 7y ago
    In the context of the question, that is not relevant. reCAPTCHA requires JavaScript as well. The question is about an alternative to reCAPTCHA. Both methods use JavaScript. I do understand where you are coming from though. And I also think this alternative is better in this regard. reCAPTCHA loads JavaScript from a third-party domain. With JavaScript spamfilters you are loading them from the first-party domain.
    - ChrisSD 7y ago
      reCaptcha has a noscript alternative with iframes and checkboxes. Once completed you have to manually copy an authorisation string to a field and submit it.
no_gravity 7y ago
I think it is best to design your own captcha around your use case. All you need to do is make the amount of work for spammers too high for targeting your site. Just recently, I added the idea of a captcha that might actually be enjoyable for users to my list of "things that should exist": http://www.gibney.de/things_that_should_exist http://www.gibney.de/things_that_should_exist The idea is to show the user a random image and ask what is on it. If the image is beautiful, that might even be fun. And there are many sites that offer beautiful public domain images. And have tags for everything in them. There probably are many other funny and enjoyable captcha ideas one could implement.
- Theodores 7y ago
  Exactly. Plus with your use case there may be other criteria, for instance, if you have an 'apply now' job application form you can take in other data such as how long it took for someone to fill in the form and where their IP address is. If you are hiring for a job in London and you are not likely to hire the office manager from Timbuktu who spends less than ten seconds uploading their CV and writing some cover letter then you can make your backend form processing not forward that email on to the HR department. Putting a timestamp in the form as a form field with it encoded is easy. On the submit side you can unencode it and come to some judgement on the matter. There is also the hidden checkbox with 'hideit' set to 1, not sure why that works but it does with a form you have written yourself, i.e. not stock Wordpress. Although we don't like Google doing their deep-stalking of the visitors, fingerprinting them in re-captcha, there is no harm in collecting a little bit about the user. The user agent, screen size and location is useful in a sales/support perspective. If someone has a posh computer that says something about them. If they are using an old copy of a Microsoft browser then that says something about them. On a general forum there can be standards of English to enforce. If someone is not using capital letters to start sentences, not using punctuation and not spelling so well then that can be flagged before they hit the 'send' button. I have done a lot of tidying up of email lists created by bots and what surprises me is how easy it is to spot the fakes. It is like the bad guys in movies and games, doing everything possible to make it easy to get 'em. If spammers did real world robbery they would carry a bag labelled 'swag', be wearing 'Groucho glasses' and a stripey jumper.
- 7y ago
jedberg 7y ago
There was a good podcast about this [0] just a couple weeks ago. They interviewed the guy who invented CAPTCHA as well as the head engineer on ReCaptcha v3. The gist of it was that in a few years, all Captchas will be useless because machine learning is too easy and cheap. The only way to defeat spam will be to use reCaptcha v3 or something like it, because those services will use what they know about you to determine if you're a bot or not, plus their own machine learning of what "normal" behavior is for your website. It sounds like ReCaptcha v3 is basically an app level IDS. [0] https://www.npr.org/sections/money/2019/04/24/716854013/episode-908-i-am-not-a-robot https://www.npr.org/sections/money/2019/04/24/716854013/epis...
- clarry 7y ago
  > The only way to defeat spam No, no, no. There are many ways to combat spam, and there is no silver bullet. "Determining whether you're a bot or not" is just one tool among many, and one that lets human spammers through, or gets too intrusive and starts blocking humans. IMO the best approach still is to focus on the content they post rather than trying to figure out who/what they are.
  - zaarn 7y ago
    Google wants you to think reCaptcha is the ONLY tool. That way they can get more user data.
- sjamaan 7y ago
  > those services will use what they know about you This is inherently user-hostile, as it presupposes tracking and identification. I don't want them to know anything about me!
  - 0xffff2 7y ago
    I'm fairly certain that Google doesn't care.
  - ufmace 7y ago
    Which is kind of the trick - there may be a point not to far in the future where it is nearly impossible to tell the difference between hostile bots and users who are just really into privacy and not being tracked.
realcr 7y ago
I had a strange idea about solving this problem: How about a micro-payment, something like $0.01, instead of solving a puzzle? In that case maybe you won't care if many bots login to your website. I think that I by this time I have the technology to make something like this work, I was wondering if this is a good solution though. What do you think?
- saagarjha 7y ago
  Payment processing is nontrivial to set up, and doesn't make sense in a lot of contexts.
  - realcr 7y ago
    What if we had a mechanism that makes this kind of setup very trivial, and also risk free for the visitor? Then would you consider this kind of solution? EDIT: Same question for tty2300 (:
    - nexuist 7y ago
      We already do with cryptocurrency. Many people are incredibly hostile to using their credit card anywhere that isn't a physical location. My parents don't even trust Amazon. They use Visa prepaid cards that they fill up at CVS whenever they have to. Arguably they're doing the right thing and we're all doing the wrong thing. Either way, micropayments are one of those "ideal world" scenarios that are unlikely to transpire anytime soon.
    - Freak_NL 7y ago
      What would prevent the bots from using the same system? The minor cost may be offset by the fact that getting a post through to a site that has almost no spam because of its filter is valuable in itself. If the bot is sophisticated enough that the stuff it posts is hard to distinguish from a human shill or shitposter it may even prefer sites protected with such a system. Useful for the vintage VIAGRA HERE link dumpers perhaps though, but those can be filtered out with a content filter.
    - 7y ago
LogIN- 7y ago
As per google "reCAPTCHA is a free service that protects your website from spam and abuse" but instead one can argue that reCAPTCHA is a service that transfer spam issue from the provider to its users, so at the end one provider will be free of spam (I guess) but all of his users will be spamed, tricked, fingerprinted and abused to actually constantly work for free for this 3rd party ant-spam service
- dmos62 7y ago
  I suppose you expand the usage of spam to also mean the recaptcha mini-games where you tick all the boxes with traffic lights in them. I agree with the sentiment. I'll add that if you don't use anti-fingerprinting or anti-tracking measures, then recaptcha catches on really quickly that you're a person, and it's not much of a bother in that case. The problem with it is that it's made for the ad-peddling web, not for the private web, or whatever the alternative should be called.
simongr3dal 7y ago
If it's a problem with spammy blog comments I would recommend to just remove any kind of input on the site and ask people to send you an email with their questions and concerns. Be sure to use a separate email and give it to readers on your about page via some language like "questions (dash) and (dash) comments (at) (this domain)". If it's for account signups just send an email confirmation link and possibly include a code in the email that has to be submitted manually as well.
tpetry 7y ago
For bots which are not specifically targeted at your page i simply add an invisible form element named url. Bots _LOVE_ to share their viagra urls. Any request which submitted an url is discarded. This trick is simple stupid and should not work but somehow the simple spam bots have not improved. This does not work for sophisticated bots (never met one) or the ones programmed specifically for your site (happens very rarely).
- enobrev 7y ago
  It's been a couple years since I've implemented one of these, but I've used this method in various forms on projects for at least a decade. Sometimes I use a hidden "email" field. Sometimes a hidden "subject" field. Hadn't thought of a "url" field, which is a great idea. It works surprisingly well.
- willio58 7y ago
  This is simple form of a honeypot but it is really ineffective. Any bot with even minimal sophistication will know to leave the hidden field empty.
  - corobo 7y ago
    You hide it with CSS, not type='hidden'
    - dev_256 7y ago
      You can still figure out it's hidden.
      - IshKebab 7y ago
        Sure but the point is that almost nobody does.
    - SquareWheel 7y ago
      It really doesn't matter. display:none or position:absolute;left:-1000px makes little difference. Honey pots only catch a small number of bots.
dessant 7y ago
W3C has published an extensive list of reCAPTCHA alternatives: https://www.w3.org/TR/turingtest/ https://www.w3.org/TR/turingtest/ W3C is requesting feedback for the document, if you'd like to make suggestions, please open an issue: https://github.com/w3c/apa/issues https://github.com/w3c/apa/issues
- dmos62 7y ago
  That's a very informative document. Privacy Pass caught my eye: https://privacypass.github.io/ https://privacypass.github.io/ It's an extension that is currently only supported by Cloudflare's CAPTCHA, that pretty much stores tokens after you complete a CAPTCHA, and the next time instead of requiring you to complete a CAPTCHA again, it will use those tokens. The point is that it does it in an anonymity preserving way. You can fork their server for a custom implementation.
  - chatmasta 7y ago
    So now the spammers only need to solve one captcha?
    - bo1024 7y ago
      And more broadly -- it's kind of funny, but the more we all just roll our own solutions to this, the less standardized the solutions, and the harder it is for spammers to scale.
  - anderspitman 7y ago
    These types of "blanket captchas" basically destroy browsing via something like Tor, right? I feel like I've seen people complain about Cloudflare's captchas when browsing ananymously.
- ukd1 7y ago
  Thanks; commented - https://github.com/w3c/apa/issues/28 https://github.com/w3c/apa/issues/28
huhtenberg 7y ago
Depends on why you need it. Captchas work well for telling humans from bots for the purpose of denying automated/scripted access. But here a simple IP-based blacklist works well, because of how many bots now live on Amazon's properties and some such. You don't need a captcha to filter out bot spam. That's a massive overkill. stopforumspam.com works well. You can combine it with a simple keyword based filter, have it tag hits with a cookie, temporarily blacklist the IP and then filter them out based on that as well. Auto-submit it to stopforumspam too. Obviously, also have whitelisting in place, e.g. to let through existing customers, previously cleared posters, etc. For bonus points, first-time posts that look OK may be put into a "shadow ban"-ish mode, whereby they are visible to the posters and mods, but not anyone else. Until they are cleared. This works equally well. The bottom line is there's no spam that doesn't try to promote something and they aren't likely to target just you, so there's always a keyword/URL you can latch onto, and it also makes sense to participate in a distributed monitoring framework to piggy-back on each other's first hits.
- dredmorbius 7y ago
  "there's no spam that doesn't try to promote something" is, unfortunately, not true. Or at least, some spam plays the long con. On sites where karma or social graph confers advantage, bots will harvest one or both through low-effort, high-payoff posts. Various disinformation and distraction campaigns may sell only confusion, discord, or volume of content. And if it's the graph itself or specific connections which matter, ossiby for hishing, recruitment, or other compromise, you'll see other behaviours. Not all media manipulation is commercial. Not by a long shot. You're fighting the last war, if not older.
weeb 7y ago
Whatever you use, please remember not everyone has good vision / hearing / dextrous mouse control. Captchas can be a nightmare for accessibility. Most of the 'clever' solutions to this will completely block some subset of keyboard users / blind users / eye gaze users along with the bots.
- cheschire 7y ago
  It's really frustrating talking to client side developers these days about 508 compliance. It feels like only one in 10 understand the concept of accessibility.
rajacombinator 7y ago
How about put your users first and don’t farm them out to Google ML training because someone told you to. Recaptcha is a cancer on the web.
- icebraining 7y ago
  Isn't that what they're doing, hence this question?
deleted 7y ago
[deleted]
nopcode 7y ago
As a user I found geetest [1] to be really friendly and much easier to use than recaptcha. I have never integrated it myself. [1] https://www.geetest.com/ https://www.geetest.com/
55555 7y ago
recaptcha v3 is invisible.
NourEddineX 7y ago
What about PoW (https://en.wikipedia.org/wiki/Proof-of-work_system https://en.wikipedia.org/wiki/Proof-of-work_system) ? It require minimum user interaction, and you will eliminate most of spammers bot, since it will lose its cost-effectiveness. You can implement something like coin-hive proof-of-work, without having to mine monero anyway
bjoli 7y ago
I have a mail server with a new address generated per post (or per comment for thread functionality) on a blog i run. People then get to mail their comments. For all reputable mail sites I let things directly through, for everything else I use a spam filter turned to 11 together with a mail-back link for post verification. I have had zero spam the last 8 years. The code is ancient and runs on an even older version of lispworks with Auth details hard coded all over the place, so I the time it would take for me to share it would be longer than to rewrite it in some hip language. Had I been lazy and not as privacy conscious I would have let Gmail do the spam filtering for me.
- pdimitar 7y ago
  Have you considered open-sourcing that and posting it on HN? I'd use it.
  - cweagans 7y ago
    > so I the time it would take for me to share it would be longer than to rewrite it in some hip language. I'm reasonably sure the answer to your questions is yes.
  - bjoli 7y ago
    I have been asked to many times, but it is part of a largish website written in Common Lisp. The codebase is written by me over 3 years with no consideration for modularity. It would be a considerable effort to break it out. There is nothing technically novel about it. Heck, python even includes an SMTP server in the stdlib. You could.probably write a PoC in a couple of hours.
Kenji 7y ago
It has come to a point that traditional text/sound captcha challenges are trivial to bypass today Don't be fooled. Text-based CAPTCHAs are still very effective, unless you're a really large target on the scale of facebook or google. If you design your own text-based CAPTCHA, it's highly unlikely that someone is gonna pull out their ML skills to read your CAPTCHAS just to spam. Too much effort. I wrote my own PHP CAPTCHA more than a decade ago and have it used ever since with virtually no modifications, and not a single piece of spam has made it through on my websites (if you don't count my friends trolling me with silly messages once in a while).
tantalor 7y ago
"went Sparta"???
awinder 7y ago
> nobody in their right mind should add a script that fingerprints users I helped vendor-select and lead implementation on a fraud solution that was an integration with SiftScience (yc-funded, https://sift.com/ https://sift.com/), which relies on fingerprinting. This was years ago but I still think about the project and how it plays with user privacy etc. I will say that -- fingerprinting as a component in fraud management is/can be highly effective. The problem is, once you get into payments fraud through bots, I think the conversation becomes way more nuanced. If you're looking for a solution to bots spamming or throwing bad data into your app, maybe that's a little extreme. But if the choice between privacy and becoming a front for credit card fraud and chargebacks, you're in a choice between who the victims of your service are going to be, and how much ill is done.
dangerface 7y ago
> nobody in their right mind should add a script that fingerprints users Fingerprinting users is no more a problem than using cookies, there are far more legitimate reasons to use these things than illegitimate. The problem is Google and Facebook using these techniques to spy on people at massive scale. Once again the problem is Google and Facebook not the internet.
- cyphar 7y ago
  The problem with fingerprinting is that it's used to track users across sites. Cloudflare's "super cookies" and ordinary ad-network cookies are both examples of fingerprinting which use cookies and could definitely argued to be bad for user privacy. A text-based challenge-response captcha doesn't fingerprint you in any way. Google's reCaptcha does -- not to mention that it uses you for free labour that I would argue should be a violation of minimum wage laws in most countries (Google hires people to do data entry for ML, so why am I being forced to do the same work for free in order to post a comment on a forum or log into a website).
rcdwealth 7y ago
Simplest way is to use filtering. ``` (defparameter spam-words '("viagra" "cialis" "v1agra" "c1alis" "tamadol" "hydrocodome" "doxycyline" "prozac" "prozca" "prizac" "doxycyclins" "anx8ety" "amytriptylone" "poker" "laxative" "anatrim" "breast" "penis" "fiorinal" "sexy" "kaspersky" "hoodia" "thyroid" "coupon.com" "vuitton" "coupon" "fetish" "famotidine" "footwear" "sweetwater" "sunglasses" "ninja" "www" "http" "cheap3ddigitalcameras.com" "aquadivingaccessories.com" "tastyarabicacoffee.com" "yourmail@gmail.com" "bit.ly" "cottonsleepingbags.com" "italiancarairbags.com" "newpopularwatches.com" "glasslightbulbs.com" "browndecorationlights.com" "fx-brokers.review" "ceramicsouvenirs.com" "xevil" "senuke" "captcha" "xrumer" "vkontakte" "апрап" "erectile" "spellingscan" "lialda" "lamborghini" "doubles your bitcoin" "pro-expert.online" "specified wallet" "selected wallet" "online casino" "multimillionaire" "win-win lottery" "lottery" "Перезвоните пожалуйста" "yuguhun88@hotmail.com" "meeting-club.online" "from2325214cv" "did you receive my offer" "Domain zone .de" "all your photos" "Pay 1 BTC" "to our bitcoin wallet" "you will be sued" "police will be interested" "hacked")) ```
- Ayesh 7y ago
  If HN had this filter, you wouldn't be able to submit this comment.
jermaustin1 7y ago
The best solution I've ever come to that didn't negatively impact my clients was generating a UUID on the server via an ajax call 100ms after page load. That UUID was stored in a cookie, and returned via AJAX and stuck it in a hidden field on the form. Server checks cookie != null and cookie == hidden field, and returned a 200 OK regardless of if it failed (used the response text for success or failure indication), and deleted the cookie. Implemented it across a network of sites ~10 years ago, and only a handful of spam had gotten through when I quit that job 4 years ago. They had been getting 10-20 spam comments per day per site.
- sireat 7y ago
  So this solution would work because most automated spam would not make AJAX calls? Something like Selenium posing as a real user would bypass this kind of protection wouldn't it ?
  - flyGuyOnTheSly 7y ago
    It works because most bots do not keep running the javascript after pageload like a real user's browser would.
- Ayesh 7y ago
  I was thinking to roll my own cookie solution as well, but this makes the form submissions impossible on multiple tabs.
- jm4 7y ago
  That will work for low end drive by stuff. Anyone motivated will have a better bot. I was running one 10-12 years ago that was essentially a headless browser. It had a JavaScript runtime and a custom DOM. It could run jquery, prototype, ajax and just about everything else that was popular at the time. I even had a custom flash runtime in there for the jackass sites with the nav in flash. These days you could just throw together a selenium script and call it a day. This kind of stuff is fine for stopping comment spam because there are so many other opportunities out there that the spammers move on to an easier target. If you need to protect against a targeted attack then it’s a lot more difficult.
saravana85 7y ago
guess most companies are now using not a robot future with any images. meaning when you click on am not a robot you will be moved to do next..
mkbkn 7y ago
I also recommend mailing the website owners who uses ReCaptcha about why it's a nuisance and stating that you and many others won't be using the site anytime soon.
- n_ary 7y ago
  Doesn’t work, they don’t care. I mailed+twitted to many among which were Dropbox/HumbleBundle/Packt etc.. most just ignore me or came back with canned responses like “we value security blah blah improves security blah blah your data...” :(
KirinDave 7y ago
Unfortunately, there aren't many good captcha systems that don't do the equivalent of what ReCaptcha does, because we're at a point where fingerprinting users is a strong signal to help identify contractors doing captchas on behalf of bots. Even some silicon valley products use captcha-breaker services. These services present themselves as sophisticated APIs but in reality they're just dispatching work to humans who accept pennies an hour at internet cafes; a competition with Amazon's mechanical turk for digital sweatshops. They're common and cheap and the tech industry feeds them. Undercutting the workforce doing the captcha busting is the only viable way to stop that. Your real alternative is to do the fingerprinting yourself.
yeppie 7y ago
For automation I would recommend ratelimiting endpoints. I personally tend to use 5 requests per IP/second along with 100 requests/minute as default and then override specific endpoints to e.g. 1 request per IP/hour. For user input I recommend keeping the first comment submitted by a new account/IP hidden until you/moderators have approved it, after which new comments from that user no longer needs to be approved before they become visible to other users.
tmlee 7y ago
Recaptcha is also blocked in China. Users there wont be able to bypass it at all to accomplish a protected task. Anyone knows of a good alternative that works there?
- Zimahl 7y ago
  Yup, this is a huge problem on my company's website. We really like reCaptcha but, alas, have to fall back to our home-rolled version because reCaptcha completely fails for our Chinese users.
gradschool 7y ago
A commenter on HN some years ago claimed a 100% success rate at blocking spam by requiring all web form submissions to be cryptographically signed. This solution struck me as stunningly elegant both by raising the standard for constructive feedback and promoting public awareness of secure communication.
- pdimitar 7y ago
  Cryptographically signed where and how? Do you mean to say that you get a full string like "a=1&b=something&c=[1,2,3]" and hash/encrypt that? Or do you encrypt each individual field? Or something else?
spydum 7y ago
Have seen these guys, met the founders a while back at AppSecUSA: https://funcaptcha.com/ https://funcaptcha.com/ It’s those puzzles/games as captchas.
NetToolKit 7y ago
For those who are interested in an alternative CAPTCHA service, we at NetToolKit are putting the finishing touches on a service that we hope to launch at the end of next month (June). The CAPTCHAs are interactive and meant to be fun for the user -- no machine learning training involved. We'd be thrilled to get some early feedback before launch, so if anyone is interested, please reach out via email or via our website (both in profile).
josefresco 7y ago
Looking at these comments (141 at the time of this post) the answer looks to be: No. I have small business clients, Google's reCAPTCHA is our best option. They aren't willing to pay for some obscure, and expensive one-off solution that might work. They just want the spam to stop. I fill out reCAPTCHAs every god damned day because I work on the web. Asking "normal" users to fill out a handful each year isn't asking that much. Maybe for your startup "rolling your own" makes sense, but not for small biz.
- NetToolKit 7y ago
  There are definitely alternatives. For example, https://hcaptcha.com/ https://hcaptcha.com/ (I have not used or evaluated them). If any of your small business clients might be interested in our new CAPTCHA service that should launch late next month, please let me know (see profile for contact information). Our pricing is projected to be $10 for 100,000 requests.
aendruk 7y ago
For the use case of blocking general web form spam, we've had good results with relying solely on IP reputation crowdsourced via AbuseIPDB: https://www.abuseipdb.com/about https://www.abuseipdb.com/about Occasionally we're an early target of a fresh IP, but we report it back to the database to help later victims. The more people contribute to such a system, the better it gets.
- packet_nerd 7y ago
  Many IPs are shared by more than one endpoint, and hundreds or thousands of endpoints sometimes share a single IP. Say a home router is compromised on an ISP using CGNAT and you block by IP, you could potentially be blocking an entire neighborhood of innocent users. Having parts of the Internet blocked for huge numbers of their customers puts pressure on ISPs to monitor and censor users traffic, which is not the direction I want to see the Internet go. IPs do not have a one-to-one relationship with users, and I feel strongly that they shouldn't be treated as if they do.
  - aendruk 7y ago
    Thank you, that's well-put. I also share that value. Another value that I hold is that the cost of bad actors shouldn't be externalized to innocent people, i.e. that it's unfair for me to pay the cost of completing a captcha just because somebody somewhere else is misbehaving. I'm not sure how to reconcile those.
techsin101 7y ago
Just ask a math question?
htrp 7y ago
> Google went Sparta with their reCaptcha be, and nobody in their right mind should add a script that fingerprints users, specially from an adtech company Elaborate?
unnouinceput 7y ago
I like those math questions captcha. Fun and I doubt a bot or even a real persona spammer will waste time on this. Make the question appear on an image instead of text and the bot will also have to do OCR on top of being wolframalpha to defeat your captcha.
- hakfoo 7y ago
  TBH, I had decent results with the most trivial "3+4=" style CAPTCHAs. I know some sites where there's even only a single, hard-coded question/answer. In the world where it's not about scaling to infinity, you're not getting targeted attacks, you're getting stupid bots that pummel every site thinking it's a WordPress install from 2007 and anything different is enough to scuttle it. By the time your service is popular enough to justify building a focused bot or tossing low-wage workers at it, hopefully you have the revenue in to finance something more sophisticated.
ptman 7y ago
https://hcaptcha.com/ https://hcaptcha.com/ ?
Too 7y ago
Unless your website is under targeted attack just putting "2+3" on a image will block 99.9% of all bots. You hardly even have to distort the image or randomize the math but doing so could help against script kiddies. Only drawback vs reCAPTCHA is you have to show the captcha all the time instead of automatically suspecting bots. If you are under targeted attack by someone more dedicated, captcha is not going to be the only defense in your book.
- massimo-nazaria 7y ago
  > Unless your website is under targeted attack just putting "2+3" on a image will block 99.9% of all bots. Absolutely. I did exactly that with a PHP script which generated images with the GD library. It definitively worked for me.
zzo38computer 7y ago
I think reCaptcha is very terrible. For HTML forms, a simple question could be used (change them sufficiently often when spam is received), or you may require the user to edit the URL manually in order to access something, based on the client IP address perhaps (which would be displayed). I also invented a protocol-independent CAPTCHA, which is also text-based, and uses SASL. You should allow the user to implement the code themself if they want to do rather than requiring that they use your code.
ak-47 7y ago
A bit late, but honorable mention: https://xkcd.com/233/ https://xkcd.com/233/

worble 7y ago

You have no idea how often I've shouted at my monitor that I would rather pay the site hoster a one-time fee than use google captcha ever again, and I really wish more sites offered this. Say what you want about the site, but 4chans pass method of payment, which just removes all the captcha's when posting and lowers the post timer, was a fantastic idea, and just works.

realcr 7y ago
I think that there is a real opportunity here. I just went over all the comments about this idea. It seems like most of the objections relate to issues with the payment medium itself, and not with the actual idea of paying instead of solving a puzzle. Some examples: > It seems more likely they are trying to steal cc details by asking for such a small payment. This relates to being afraid using a credit card online against a non credible seller. > Payment processing is nontrivial to set up This one is about the technical difficulty (And possibly also regulation) for setting up payment processing > regions where it is uncommon to have means to interface with payment processors. This one is about inaccessibility to payment means, which also relates to a problem with the payment medium. > You could use JavaScript cryptocurrency mining instead I assume that bo1024 suggested this because he implicitly believes that setting up a payment processor will be more difficult, and give bad experience to the user. The only objection I have seen which is more specific to actually recognizing humans is this one (by Freak_NL) > What would prevent the bots from using the same system? First, I think that it should make operating bots on a wide scale much more expensive. Second, if you make money out of bots coming into your website, would you really care that they are bots? I assume that the reason someone would block bots from the first place is that bots harm their money making business. If you had a system to collect this money back, shouldn't this considered as a problem solved? I am working these days on a decentralized payment system that supports micro-payments. (I promise there are no blockchains inside). Solving the captcha thing can be a very useful use case. I will be happy if anyone wants to join forces on this one (My email is real (at) freedomlayer (dot) org).

mikojan 7y ago

So now you're filtering out bots and people with disabilities?

vbezhenar 7y ago
You can set placeholder with something like "leave it empty".
icebraining 7y ago
Nope, screen readers (usually?) ignore elements with "display: none". To be sure, you could add aria-hidden="true", which I'd guess most bots don't recognize. http://alistapart.com/article/now-you-see-me/ http://alistapart.com/article/now-you-see-me/
- ghusbands 7y ago
  Many don't just use display:none but position text off-screen or make it tiny or use sizing and overflow:hidden. I've seen a blind user be tripped up by these, so yes, it often also filters out disabled users.
  - icebraining 7y ago
    Many people use tools incorrectly, that doesn't mean you shouldn't use those tools, you just have to be aware of the problem, which everyone in this (sub)thread now is.
  - londons_explore 7y ago
    A decent screen reader will look at the final rendered Dom and obviously cull items off-screen, the same colour as the background, or 1px high. If it doesn't, its a bug in the screen reader.
    - ghusbands 7y ago
      No such screen reader exists, so you could claim it's a bug in all of them, but it's not a useful position to take. Screen readers are tricky enough to write without having to second-guess people who are trying to hide-but-not-hide text.
    - detaro 7y ago
      There's quite clear standards on how screen readers are expected to interact with the DOM. What you claim is in none of them, and the techniques you mention are explicitly mentioned as something authors of webpages should avoid/mark up correctly for screenreaders.
corobo 7y ago
Unfortunately I've not given thought to people with disabilities in this case. It's an oversight and that's on me. I wonder if bots are smart enough to figure out aria-hidden="true". Possibly empty it out using javascript on submit. I would guess a bot not using CSS would also not be using javascript? Unsure, would need testing.
- kaybe 7y ago
  Could we just tell the people with screen readers to ignore it?
  - corobo 7y ago
    As a fallback I imagine so yeah. I'm trying to think about what would be going on at the time, wondering if unexpected instructions inside a form would be confusing. Disclaimer: I don't know how a screenreader would present this, example only "Form entry. Input name. Input email. Ignore this field it's for spambots. Input url. Submit" -- In this case does the message more naturally apply to email or url? I'd imagine there'd be a pause after input email (to wait for the input)? I need to set up a screen reader :)
    - chronogram 7y ago
      And then hide the warning using CSS too, so it gets picked up by screen readers just like the hidden input!
benburleson 7y ago
Should we put the onus on screen readers to not read elements that would not be displayed when the page is rendered in a browser?? Admittedly, I am not familiar with screen reader standards, but my gut feeling is that they are doing their users a disservice if they are not representing what browser users are seeing as similarly as possible.

colinbartlett 7y ago

If you work in the Ruby ecosystem at all, there is a gem called invisible_captcha that does just this. It beat back most of the bot signups we were getting, though some still slip through occasionally. https://github.com/markets/invisible_captcha https://github.com/markets/invisible_captcha

alexpapworth 7y ago
Awesome! Love a good Ruby solution. Please keep these coming :)
ghusbands 7y ago
Note that that library is not accessible and is hence illegal to use for most websites in the US. I've raised an issue on the project [1]. [1] https://github.com/markets/invisible_captcha/issues/52 https://github.com/markets/invisible_captcha/issues/52
- oauea 7y ago
  Blind people are basically bots. If they're not viewing and interacting with the screen it's hard to tell the difference.
- benburleson 7y ago
  Whoa, it is actually illegal to make "not accessible" websites?
  - aaomidi 7y ago
    Yeah it's been the only way to make sure people with disabilities aren't left behind.
  - terminous 7y ago
    In the US, it is if you're an incorporated business serving as a "public accommodation" as defined in the Americans with Disabilities Act, which includes hotels, restaurants, theaters, storefronts. If it is just your personal blog, it doesn't apply. It is the same law that requires storefronts to have wheelchair ramps, but not homes. See Gil v. Winn-Dixie: https://scholar.google.com/scholar_case?case=6744502269111605689&q=257+F.Supp.3d+1340&hl=en&as_sdt=20006 https://scholar.google.com/scholar_case?case=674450226911160...
    - ghusbands 7y ago
      More than that; as someone else linked [1], if you're in a business employing at least 15 people (more than half the year), you are also required to have your site be accessible. You don't have to be serving as a public accommodation, for that. [1] https://www.businessnewsdaily.com/10900-ada-website-requirements.html https://www.businessnewsdaily.com/10900-ada-website-requirem...
  - MarkyC4 7y ago
    Ontario, Canada has a similar (maybe a bit more relaxed) law: https://www.ontario.ca/page/how-make-websites-accessible https://www.ontario.ca/page/how-make-websites-accessible
  - r00fus 7y ago
    https://www.section508.gov/manage/laws-and-policies https://www.section508.gov/manage/laws-and-policies They even have a compliance tester.
- graton 7y ago
  Are most websites run by businesses which employ 15 or more full-time employees? To be illegal the website must be run by a business which employs 15 or more full-time employees. Or the business is some form of public accommodation like a hotel. From what I have read. Of course it would be better to make sure the website is accessible, but I'm mostly commenting on the statement that it is illegal. https://www.businessnewsdaily.com/10900-ada-website-requirements.html https://www.businessnewsdaily.com/10900-ada-website-requirem...

ghusbands 7y ago

Be very careful how you do this, unless you want to exclude blind users. I've seen a blind user have an online form silently fail at them because they filled in a field that wasn't visible. Using display:none applied indirectly via CSS is probably reasonably effective against bots and won't interfere with screen-readers.

mrighele 7y ago
Are there tools or guides to help test how websites "appear" to people with disabilities ? It is difficult to design something if you don't have an idea of what will be the outcome, but I wouldn't know which software is used by (e.g.) a blind user, let alone how I would use it.
- kqr 7y ago
  I couldn't find anything like it, which surprises me. It should be in the interest of companies selling screenreaders that the web is accessible with them. Creating a service where you can submit a link and it shows you a textual representation of how the screenreader sees the page would be immensely useful.
  - y4mi 7y ago
    uh, there are a lot of services that do automated scans. These obviously arent completely failsafe, but still give an indicator. just google relevant keywords https://www.google.com/search?q=accessibility+scan+website https://www.google.com/search?q=accessibility+scan+website and if you want the full experience: just enable the screen reader and try to use your website with it. /edit: and i almost forgot: chromes build-in Audit tool in the Developer Panel includes some Accessibility tests as well
    - kqr 7y ago
      These scans are in my experience always just that: they read the source and match it against a set of common anti-patterns. I was talking about something that would tell me how common commercial screen readers interpret an arbitrary new construct. If you have a specific reference to something else, please send me a link! Screen readers often cost significant amounts of money, and are not trivial to "just turn on".
      - kiliankoe 7y ago
        I believe y4mi was referring to screen reading software, not a physical device. Like VoiceOver on macOS.
- RegW 7y ago
  You could start here: https://www.w3.org/standards/webdesign/accessibility https://www.w3.org/standards/webdesign/accessibility
  - RegW 7y ago
    I'm sorry - did I offend someone. Or did limited mobility cause a click on the fiddly and untitled downvote link instead of following the link :-)
- jacekm 7y ago
  Simply try to browse the page without using mouse, just the tab key. The next step would be to use screen reader, NVDA[1] (Windows) or Apple VoiceOver (MacOS). There are automated testing tools, but they don't cover the whole spectrum of problems. Nevertheless you can try * WAVE (Chrome or FF plugin, https://wave.webaim.org/extension/ https://wave.webaim.org/extension/) * AXE (https://www.deque.com/axe/ https://www.deque.com/axe/) * AChecker (https://achecker.us/checker/index.php https://achecker.us/checker/index.php) * Funkify (Chrome plugin, tries to emulate various disabilities) * Lighthouse in Chrome Dev Tools also checks some accessibility rules The full list of things that you need to take care of: https://www.w3.org/TR/WCAG21/ https://www.w3.org/TR/WCAG21/ (it's huge I know, it takes 5-7 days to test everything from this list)
  - mises 7y ago
    Please do the tab test. Some of us do this even if we don't need accessibility tools, as it can be faster (those who create forms which cannot be tabbed through are evil).
- joelhaasnoot 7y ago
  You can switch any iPhone to accessibility mode. Those with vision impairments love their iPhone because generally "it just works"
- dbjorge 7y ago
  There are lots of options for automated accessibility scans, but there are many common accessibility issues that aren't feasible to test for with automation. My team at Microsoft recently open sourced a tool called Accessibility Insights (https://accessibilityinsights.io https://accessibilityinsights.io). The web version is a chromium extension that includes both automated scans and also a guided assessment option that leads you through how to test for and fix the stuff that has to be found manually. This is the tool Microsoft pushes its own teams to use as part of their release processes.
- ilikehurdles 7y ago
  Surprised no one has mentioned this yet, but pa11y was what a large enterprisey company I used to work at tried to adhere to. http://pa11y.org/ http://pa11y.org/
- tongo 7y ago
  Everything everyone else has said. However, if you don't have a budget for this, you can do a few basic things that will greatly improve the usability of your site. First, can you navigate your entire site without using a mouse (including any widgets, forms, embeded stuff)? You should also have a "Skip to main content" button that is the first element you hit when you tab into your page. Next, download the NVDA screen reader, which is free, turn off your monitor (or close your eyes), and navigate your site using it. I recommend using FireFox for this. Finally, use a color contrast analyzer plugin for your browser to ensure you have enough contrast between all of you elements. From there, you can review the WCAG 2.0 spec to get into the fine details. If you have the budget, hire a consultant/contractor. What I described above doesn't make your site pleasent to use for a disable person, just usable.
tragicpapercut 7y ago
Can confirm, formerly used a spin of this technique and had to stop to better support blind users.
NotPaidToPost 7y ago
On the face of it this looks like an issue with screen-readers reading the page simplistically. An element that is not displayed should not be 'displayed' by screen-readers either.
xofer 7y ago
I've used this technique with a label that said "Leave Blank", also hidden with CSS. Seemed to work great, but now I wonder.
tongo 7y ago
We do this, but we've labeled our field very explicitly with something along the lines of "Please leave this field blank, it is for SPAM control.". We also provide a hidden error message if it is filled in to alert the user that it really should be blank. It's also always place this field after the Submit button with the idea that a user with a screen reader would never make it that far. Bots still see it and add it to the post request since I don't think they care about the order of the form fields.
zzo38computer 7y ago
But then what if you disabled CSS or use Lynx? (You should also add text next to it to say don't use it, and hide that text as well sa the form field. Then, if the form field is visible for any reason, the user will know not to use it.) (Using scripts for it also applies, if you have scripts disabled or are using Lynx or something that does not implement JavaScripts.)
mbillard 7y ago
I wonder if using aria-hidden on the input field would work or if the bots would also ignore it.