33 ms·

The Trouble with Tor

323 points by jgrahamc 11y ago

hippich 11y ago
Probably not really suitable for Cloudflare scale, but for others who are forced to use CAPTCHA, consider https://hashcash.io/ https://hashcash.io/ for easy Proof-of-Work integration.
deleted 11y ago
[deleted]
sajid 11y ago
Great blog. Some of the best technical writing out there.
nxzero 11y ago
"Trouble with Cloudflare" Treating TOR traffic the same as non-TOR traffic makes no sense; read the main link for confirmation they do. Case in point, and for starters, STOP repeatively requiring a user from a session to keep passing "I'm not a robot" tests. Set a global cookie that's valid for the session, across all of Cloudflair's network, and honor it. If the "I'm not a robot test" doesn't work unless it's repeatively given, then that is the problem, not TOR. Please address this issue; thanks.
- x1798DE 11y ago
  That wouldn't really help much, because then a bot owner just needs to solve one CAPTCHA, receive the "I'm not a robot" cookie, then hand it over to their bot.
  - nxzero 11y ago
    Right, which is the issue, CAPTCHA's don't work.
    - noxToken 11y ago
      Disagree. They don't work if you only show one CAPTCHA. Your proposal defeats the entire purpose, because it only takes human interaction once to defeat CAPTCHA over CloudFlair's entire network. CAPTCHA isn't a silver bullet, and I don't think anyone is lauding it as the end-all be-all to stop spammers and malicious activity.
      - nxzero 11y ago
        Have you used TOR and experienced what Cloudflair does? The server 3-10 tests per page requested in some instances collecting massive amount to training data; I'd in fact be suprise do if Google doesn't place an incentive on collecting as much data as possible; likely enough to ID a user 99% of the time based on the input provided by the user. As for it not being a silver bullet, it's well known that CAPTCHA don't stop bots, but mine data from people. There's not a single Google "I'm not a robot" that can't easily be circumvented, I just know most TOR users aren't able to and Google & Cloadflair are exploiting this.
- alex_duf 11y ago
  If you set a global cookie, then someone intercepting the public traffic (think NSA, GCHQ,...) can identify what the user is reading. You just killed anonymity.
  - nxzero 11y ago
    Global cookie is a session, if the user wipes the cookie, resets the TOR connection, etc. that's their issue. TOR is not designed to hide sessions, nor would setting a global cookie break anonymity unless the user doesn't understand how TOR works. All exit nodes are watched and session device fingerprints are correlated with or without a global cookie.
    - ikeboy 11y ago
      If nobody set a global cookie, a passive attacker cannot correlate different tabs using different circuits. See https://trac.torproject.org/projects/tor/ticket/3455 https://trac.torproject.org/projects/tor/ticket/3455 and https://www.torproject.org/projects/torbrowser/design/#identifier-linkability https://www.torproject.org/projects/torbrowser/design/#ident...
      - nxzero 11y ago
        If the same global cookie is accessible via two circuits, that a bug in a product that uses TOR, not TOR; I personally go above and beyond simply creating a new circuit, never open two circuits at the same time or boot, limit TOR sessions to single use, and locally compartmentize data per session, etc. TOR is not plug in play, it takes effort and discipline, and will never be a fully automated solution.
        ikeboy 11y ago
        Yes, TBB on default settings is vulnerable to associating multiple tabs (if I'm reading the link above right), if an adversary sets a shared cookie. That does not mean it's ok for someone to set a shared cookie. The possibility of exploitation does not mean exploitation or making exploitation easier is fine.
amelius 11y ago
CAPTCHAs are essentially a broken idea: it is easy for an attacker to send the CAPTCHA image to another website and ask users to fulfill the CAPTCHA for a completely unrelated goal. This trick has been used in the past on certain pr0n websites (users are allowed to see a picture only after they complete the CAPTCHA). Also, one could use a mechanical turk service to circumvent CAPTCHAs.
- basch 11y ago
  reCAPTCHA does things like track your mouse movement and a bunch of other hocus pocus.
  - amelius 11y ago
    > and a bunch of other hocus pocus Sounds like security by obscurity. By the way, some people use a track pad (with a stylus), where the mouse can jump discontinuously.
    - Klathmon 11y ago
      They aren't taking any one thing at face value, but are combining them to get a better picture. You might use a trackpad, so you'd "fail" that test, but your useragent is normal, you've been seen before with those cookies, and your IP is good so you are fine. But if your IP is a known "bad actor", your useragent is something never before seen, your mouse movements are abnormal, and your keyboard inputs are instant, well all of that combined means you are getting blocked.
      - amelius 11y ago
        > But if your IP is a known "bad actor", your useragent is something never before seen, What if I install a new computer on an IP address freshly provided to me by my ISP? Or what if I just open a new incognito window? Will I get blocked? > your mouse movements are abnormal, and your keyboard inputs are instant It seems to me these are really easy to fake programmatically.
lazyjones 11y ago
What's the rationale for this weak crypto in Tor? Tor uses hashes generated with the weak SHA-1 algorithm to generate .onion addresses and then only uses 80 bits of the 160 bits from the hash to generate the address, making them even weaker. Other weaknesses include use of RSA-1024, people have been complaining to no avail since at least 2013: http://arstechnica.com/security/2013/09/majority-of-tor-crypto-keys-could-be-broken-by-nsa-researcher-says/ http://arstechnica.com/security/2013/09/majority-of-tor-cryp...
- petertodd 11y ago
  This isn't actually a vulnerability yet, as SHA-1 is only known to be vulnerable to collision attacks (where you try to find two messages with the same hash) rather than pre-image attacks (where you try to find a message with a specific hash); almost no hash functions have ever been found to be vulnerable to pre-image attacks: https://github.com/zooko/hash-function-survey/blob/master/preimage-attacks-color.rst https://github.com/zooko/hash-function-survey/blob/master/pr... Secondly, the issue is moot because generating collisions on 80 bits takes just 2^40 work - not hard at all. Basically what that means for Tor, is that while it'd be pretty easy for a Onion site operator to generate two keys corresponding to the same .onion address, an _attacker_ still has to do 2^80 work to attack a site by generating a key with the same Onion address. While that's not great - 2^128 work is considered "standard" in cryptographic work - 2^80 work is still hard enough that there are probably cheaper ways of attacking Onion sites (for reference, the cumulative total work done by all Bitcoin network miners in the entire history of Bitcoin is about 2^80 hashes). As for the 1024bit pubkeys, I'm not sure what the status of that is; from what I hear Tor is actively working towards a Onion redesign that will fix these issues, and longer pubkeys may have already been fixed.
  - zmanian 11y ago
    Tor 224 proposes moving hidden service addresses to full ed25519 keys rather than truncated hashes of RSA keys amongst other things. https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.txt https://gitweb.torproject.org/torspec.git/tree/proposals/224...
  - 11y ago
Klathmon 11y ago
"To make sure our team understood what a pain CAPTCHAs could be, I blacklisted all the IP addresses used in CloudFlare's office so our employees would need to pass a CAPTCHA every time they wanted to visit any of our customers' sites." Say what you will about cloudflare, that's an impressive move.
- jgrahamc 11y ago
  What he didn't say was that he did it without warning and it lasted 30 days. Imagine what that was like for the technical support and customer success teams who were helping customers with their sites.
  - deleted 11y ago
    [deleted]
  - brianwawok 11y ago
    I am a strong believer in dogfooding. If it wasn't a pain you felt, it wouldn't be prioritized to be fixed.
    - sievebrain 11y ago
      Dogfooding is great, if there actually is a fix to the issue at hand. If the issue is "anonymous internet access gets abused a lot" then I'm not sure what a tech support guy is meant to usefully contribute to ending the dogfooding pain.
      - throwaway7767 11y ago
        > Dogfooding is great, if there actually is a fix to the issue at hand. If the issue is "anonymous internet access gets abused a lot" then I'm not sure what a tech support guy is meant to usefully contribute to ending the dogfooding pain. The problem being solved was not "anonymous internet access gets abused a lot", it was "the mechanism we use to combat abuse is too aggressive for some segments of our users and thereby denying service to actual humans, and these challenges are invisibile to our engineers and employees because they don't browse on connections that trigger the system". As the blog post shows (and this is backed up by my experience as a user of Tor) CloudFlare significantly improved its handling of this in response. So I would say this is a success. I am sure that pressure from employees, including support guys, caused this long-running issue to finally get the attention it deserved.
belorn 11y ago
One should always be doubting when it comes to security features that do not present the benefits in a clear way. The article for example claims that 18% of global email spam comes from harvested emails that is collected using tor, but is those 18% exclusively using tor? Do uses who publisher their email address on their website that is hosted through cloudfare see a lower number of spam? It should be a fairly easy thing for cloudfare to test, while also testing vulnerability and login attempts. As an aside, it would also be interesting to see if there is a quality vs quantitative differences in the malicious activity (ie, if serious attempts are done through botnets, and script kiddie activity is done through tor). The last a final test in order to verify a security measure that has such a high cost as this one, is to ask if its has any meaningful impact to the end result. A website with 10000 vulnerability scans per day is not going to be meaningful improved if it was reduced to 5000 per day, even if that is a 50% reduction. If there is a known vulnerability, the site is going to get hijacked either way.
wolfgke 11y ago
This is why people should browser a lot more over Tor, such that the relative amount of malicious traffic is reduced and thus operators will not be able to uphold the argument that Tor is often malicious traffic anymore.
- seabee 11y ago
  Unfortunately, it won't work. 1. It's a charity tax; you have to convince people to incur the cost of Tor (i.e. CAPTCHAs everywhere) for activities that don't require Tor. 2. You can't neutralise a poison by diluting it. Firstly, from the operators' POV, if there's a widespread agreement that people use Tor even though they don't need to, then they know voluntary users can be pressured not to use Tor through sheer inconvenience. Even if you wanted to boycott a service that blocked Tor, it's notoriously hard to make good on that threat unless you wield a lot of power or annoyed a very large number of people. So the consequences are minor. Secondly, the percentage of malicious Tor traffic is a red herring. What operators care about is the origins of malicious traffic. If 50% of your attacks come from one particular country (or Tor) and the cost of losing that traffic is less than the cost of that malicious traffic, there is a real incentive to block that traffic. Combined with the first point, the cost of losing voluntary Tor users is insignificant if they can easily choose not to use it.
  - wolfgke 11y ago
    > 1. It's a charity tax; you have to convince people to incur the cost of Tor (i.e. CAPTCHAs everywhere) for activities that don't require Tor. People are willing to invest personal ressources for charitable purposes. Why not here? People are willing to fight against discrimination. Why not against discrimination of Tor users? > 2. You can't neutralise a poison by diluting it. There is also poisonous traffic from non-Tor adresses. > Combined with the first point, the cost of losing voluntary Tor users is insignificant if they can easily choose not to use it. People would strictly avoid restaurants that don't serve coloured people. Why don't they avoid services that don't serve Tor users?
    - Vendan 11y ago
lake99 11y ago
HN is the only place I bother solving CAPTCHAs. For everything else, Firefox (Tor Browser) has plugins to get a copy from arhive.org, archive.is, or google-cache. So if the page asks me to solve a CAPTCHA, I don't visit them. This could be a benefit for the website (lower server load) or a harm (fewer people appear to be reading their content, fewer people see their ads). Whatever the case may be, I'm caught in the crossfire between crackers and servers. I don't care about their war at all. As far as I'm concerned, I'm winning. Cloudfare wanted me to solve a CAPTCHA to read their article. I tried to archive it, but arhive.is already had a copy of it. This happens to me quite often. So, obviously, I'm not the only one who has figured out a way around their war. > Security, Anonymity, Convenience: Pick Any Two Nah, I usually have all three.
- scrollaway 11y ago
  > I usually have all three ... > Firefox (Tor Browser) has plugins to get a copy from arhive.org, archive.is, or google-cache. So if the page asks me to solve a CAPTCHA, I don't visit them. You don't have convenience.
  - Sir_Cmpwn 11y ago
    A one-click measure to circumvent a captcha is pretty good convenience-wise imo.
    - scrollaway 11y ago
      Sure, but google cache/archive pages often lack some images, have broken javascript, etc. Additionally, there's a huge difference between "point and click", and "point, click, click again, look for the plugin, another click, do this for every page".
      - lake99 11y ago
        > "point, click, click again, look for the plugin, another click, do this for every page". Honestly, I don't know how I can make the process as complicated as you described it, even if I wanted to. In reality, it is no more complicated than right-click, open in new tab.
rnhmjoj 11y ago
Why is everyone obsessed with knowing whether the user is a bot or an actual person? What differences does it make? A bot it's not inherently malicious, there are thousand of legitimate use cases: a bot may be downloading content as part of a script or some application, checking updates or creating a cache. Traffic generated by bots should not be blocked per se. There are certainly lots of malicious bots scanning for vulnerabilities, DDoSing sites and so on but this applies to people as well. I think human generated traffic may have priority but blocking bots entirely is nonsense: ultimately the user agent is always a "bot" acting on behalf of an actual person: by blocking this traffic you may always break some user workflow.
- jeffasinger 11y ago
  CAPTCHAs are useful when you want to rate limit something to an extremely low rate, like for example attempting to login with a username and password.
  - rnhmjoj 11y ago
    You could slow down the responses or rate limit the requests, with no need to completely block automatic logins.
    - progval 11y ago
      You would have to limit it per IP address if you do not want to after all clients. So it would not be effective against someone who can use many IP addresses (eg. with Tor)
    - jeffasinger 11y ago
      If you are a valuable enough target, this is not an option. IPs are cheap, if you let someone try 20 times in an hour before banning an IP, there are targets that people will cycle through IPs that quickly for.
    - dredmorbius 11y ago
      Tor largely prevents most approaches to this. You'd need some way to provide a ticket to a given client, check for it later, and, preferably, ensure anonymity over time. I've just posted top-level in this thread listing two projects of which I'm aware that provide this, though as experimental protocols only. I've been mildly agitating for further development of such tools. Looks as if CloudFlare are working in a similar direction, which I see as positive.
monoclonal 11y ago
Worth noting that a surprisingly common amount of sites have other sections of their site which are not routed through Cloudflare. Look for instances of DNS records like this: admin.example.com Such a record is usually not routed through Cloudflare because the last thing a webmaster wants is to solve captchas for their own website. They don't however care much for their visitors if they're subjecting (possibly a substantial amount of them) to captcha solving nonsense. The content in the non-cf sections of a site can still be accessed because the webmaster is lazy and didn't care to check if a visitor can do a DNS DIG on all their DNS records. Or you can simply use TOR pluggable transports to pretend you're Googlebot, and also hide all your traffic in Google-like traffic. I would reserve this for rare cases as there are people in censorship prone countries who really need this bandwidth :)
- besselheim 11y ago
  The pluggable transports are for connections from your Tor client into the Tor network, not connections from exit nodes to the rest of the Internet. Cloudflare (or any destination host) would still be able to detect your connection as originating via Tor.
  - monoclonal 11y ago
    There are innumerable ways infact to spoof the fact you're not using TOR to a website, and you can read up on these in the TOR documentation. Ideally you're looking to use TOR as the first hop, and then you dial into the wider Internet with a VPN, or as I mentioned: Using various Google services to camouflage traffic instead of a VPN. This is where pluggable transports come in, because Google doesn't like TOR, so you want to choose how you're connecting to Google, and get to traverse the TOR network to find an optimal route.
    - besselheim 11y ago
      Pluggable transports are intended to stop your local ISP detecting or blocking your connection to Tor: https://gitweb.torproject.org/torspec.git/tree/pt-spec.txt https://gitweb.torproject.org/torspec.git/tree/pt-spec.txt While I agree that you can mask your exit from the Tor network via an additional proxy or VPN, that's not the role of pluggable transports. They're only for connecting in to the Tor network, not out from it.
thraway235 11y ago
anyone else concerned that the captcha's offer an ability to de-anonymize someone completely? for example having a backend algo offer certain captcha's that show up only in certain areas of the world? I feel like this is entirely possible and is part of the reason I will not complete and captcha's moving forward.
- jnagro 11y ago
  definitely a big problem. and its Google who has all the data.
- TorKlingberg 11y ago
  When using Tor, reCAPTCHA cannot tell where you are.
barrkel 11y ago
I have never successfully completed a captcha served up by cloudflare (and thus Google) on Tor. They are fiendishly difficult to the point I suspected the mechanism is broken.
- Sir_Cmpwn 11y ago
  +1 I just give up immediately when I see a Cloudflare captcha page on Tor.
- throwaway7767 11y ago
  > I have never successfully completed a captcha served up by cloudflare (and thus Google) on Tor. They are fiendishly difficult to the point I suspected the mechanism is broken. Have you tried recently? They have gotten way better. There was a time not long ago where I would have agreed with you (the CAPTCHAs were literally impossible for a human in most cases). CloudFlare has toned the CAPTCHAs down a lot recently, they're now presenting image classification tasks (select street signs/bodies of water/storefronts/cactuses/...), and in my experience, my success rate is close to 100% on those. Admittedly they still make me solve three of them and they get tiring, but at least they're not completely cutting off access to Tor users anymore.
- buro9 11y ago
  I use the audio captcha. Works every time. The ones that are impossible for my are the street signs, and after that all of the cultural ones (I see US captchas and when asked to select a sandwich or a recreational vehicle I'm doomed to not complete them). An interesting side effect of failing a captcha is that to Google this looks like proof the captcha is working, that you're likely to be a bot, and that they should definitely give you the hard captchas. As such, if you cannot complete a captcha the chances increase that you must now complete multiple difficult captchas. The audio captcha is delightfully simpler though it does take a moment longer to complete.
  - MaddoScientisto 11y ago
    Doesn't the audio captcha work only if you have javascript enabled? If you enable javascript on TOR then you are doing it wrong. I could be wrong though
tech-no-logical 11y ago
I don't use Tor daily (although I run a small exit node), but I do surf via a VPN. cloudflare's captchas drive me to the brink of insanity, I see at least 50 each day...
- brianwawok 11y ago
  Have you considered setting up your own VPN? That would make the ip "clean" and get less captchas, right?
  - amdavidson 11y ago
    > Have you considered setting up your own VPN? That would make the ip "clean" and get less captchas, right? And would negate any anonymity offered by using a VPN.
    - mathrawka 11y ago
      Tor : Anonymity :: VPN : Privacy VPNs do not provide anonymity.
      - startling 11y ago
        I think you misunderstood -- I parsed it as "And using a VPN would negate any anonymity offered (by using Tor)".
- Zarel 11y ago
  Cloudflare says: > With most browsers, we can use the reputation of the browser from other requests it’s made across our network to override the bad reputation of the IP address connecting to our network. For instance, if you visit a coffee shop that is only used by hackers, the IP of the coffee shop's WiFi may have a bad reputation. But, if we've seen your browser behave elsewhere on the Internet acting like a regular web surfer and not a hacker, then we can use your browser’s good reputation to override the bad reputation of the hacker coffee shop's IP. I occasionally use a VPN, but I've never gotten a CloudFlare captcha. Is it possible that you might be doing something else other than just using a VPN, such as blocking cookies?
fosco 11y ago
captcha's are evil. unfortunately I do not trust tor.[0][1] I'm unsure the internet will ever be anonymouse unless large completely private networks start gaining popularity. [0]http://fusion.net/story/238742/tor-carnegie-mellon-attack/ http://fusion.net/story/238742/tor-carnegie-mellon-attack/ (11/2015) [1]http://www.theguardian.com/technology/2014/jul/22/is-tor-truly-anonymising-conference-cancelled(7/2014) http://www.theguardian.com/technology/2014/jul/22/is-tor-tru...
beeboop 11y ago
In addition to CAPTCHAs, why not just have a button that runs some JavaScript that completes a proof-of-work similar to what mining bitcoins does? You could make it only take 5 seconds on a modern laptop CPU, about as long as it'd take to enter the CAPTCHA anyway, but it'd potentially be a very large road block for spammers/DDOSers. For those on phones, you can still opt for CAPTCHA if you don't want to kill your battery.
- JoeAltmaier 11y ago
  Love it! I hope almost all authentication will become automated. My personal client machine is much better at proving who it (I) am that a human.
- mikeash 11y ago
  I did this with the comments box on my blog. I don't think it's technically effective, because doing proof of work in JavaScript is orders of magnitude slower than doing it natively (especially with GPU acceleration). It works well enough for me, because I'm not a big enough target, but it wouldn't be a major obstacle for someone who really wanted to cause havoc. It would be interesting to see how fast you could make the JavaScript code. I'm sure my version is just terribly unoptimized. The requirement to support the least common denominator will present a major problem, though.
  - tne 11y ago
    I would look at asm.js, do some tests with major browsers and beef it up. It would make it painful under user agents that don't JIT (or compile AOT) by leveraging asm.js conventions (many do though). If the wasm effort works out (it's looking like it will), it would hopefully alleviate the issue you present entirely and make this solution viable in a very sane way.
- pfg 11y ago
  Spending 5 CPU seconds to submit one comment might be significantly cheaper for a spammer than paying someone to solve the captcha for them. Additionally, specifically with Tor users, you can expect a large chunk of the user base to have JavaScript disabled completely. You can do many things with JavaScript that could be used to build a browser fingerprint, so someone who's already using software to browse the web anonymously is very likely to disable that.
mdip 11y ago
I played around with Tor a couple of months ago when it hit the headlines again, just to see what it was all about, and the experience was awful. As an exercise, I decided to fire up Tor browser today just to see if it's gotten any better. It's worse. Here's what I found: Large Image based CAPTCHA: Tor is slow. I'm on a 75/25Mbps internet connection and it loads images similar to 24k dial-up. The CAPTCHA I was presented with was the highest bandwidth CAPTCHA I've ever seen. I was given a 9 pictures and needed to select "Bodies of Water". Each click yielded a new square. I had to wait for 4 additional images to load before I could click "Validate". This took over a minute. Then, repeat, this time "Store Fronts", which were hard to discern (is it a blurry Apartment Building front or Store Front?). I received a connection error on one site so I had to repeat the process. With Javascript features turned off, it was a little easier, but included the extra step of having to paste a Base64 encoded string into a text box which failed twice. Every site I tried gave me this CloudFlare CAPTCHA page. One of the sites I pulled up had no images. I set my privacy settings to the least protected, enabling JavaScript and HTML5, assuming this was the problem. Nope. They used images from another site and I had to grab the image URL and paste it into a browser to see what was going on. It was yet another CAPTCHA. A few minutes later, the previous site displayed images properly. On to Google. Privacy settings are still at the weakest setting. Type "Google" into the search engine and I get a "wavy text in an image" CAPTCHA. This loads quickly and is easy to answer, but just results in another CAPTCHA. I gave up after 10 tries. Bing, Yandex and Yahoo all worked with Yandex only presenting a CAPTCHA once after the third search I did (simple, like Google, but worked). This is a terrible experience for people seeking to get around oppressive governments. While I applaud CloudFlare for dogfooding their CAPTCHA system, I doubt they did it in a way that truly simulates the experience via an extraordinarily slow internet connection which is what I ended up with when using Tor Browser. I wonder how much slower things would be if my internet connection was 1Mbps or being interfered with by government infrastructure. I understand the trade-off between securing a site from "evil traffic" that is more likely to originate from a Tor exit node, but why must they use such a bandwidth intensive CAPTCHA? A browsing experience that would have taken seconds to complete took me almost 5 minutes (and a lot of frustration) not including actually consuming the content I was looking for. Are the text in image based CAPTCHAs not good enough for this task? Are there other reasons I'm missing?
dev1n 11y ago
For users who can afford it, routing a vpn through tor solves the captcha issue with cloudflare. Also adds an extra layer of security
- amdavidson 11y ago
  Can you elaborate? I cannot imagine how Cloudflare could distinguish VPN traffic routed through Tor and standard traffic routed through Tor. The only difference is a hop on the front end, no change to what comes out the exit node.
  - dev1n 11y ago
    if you setup an access point to route all of your traffic through tor, then connect to your VPN through that access point, your IP is the VPN IP, not the exit node.
    - icebraining 11y ago
      What the point of doing that over connecting directly to the VPN? Seems like the benefit granted by Tor (avoiding leaking who connected to the VPN) would be negated by the fact that there are now payment records from you to the VPN.
      - dev1n 11y ago
        If you pay with BTC, prepaid gift card (paid for in cash) etc.. then there are no payment records from you to the VPN. The benefit of this is that the VPN provider doesn't know who you are because you are accessing the VPN through tor. Yet the VPN provides you a stable IP that won't be CAPTCHA'd like most normal tor exit nodes are. Edit: This is good if you are trying to maintain an Internet profile (i.e. Facebook, twitter etc.) that isn't tied to your true identity.
        jobbleobble 11y ago
        But you are losing out on some anonymity here. The VPN provider may not know who you are but you are consistently making access with the same user ID and from the same IP. Your activity can be correlated to that account and that IP. If that's not your aim (like you say - being signed in to the same facebook account all day suffers you the same problem) then this isn't an issue. But this isn't what a lot of tor users want tor for.
zhte415 11y ago
My trouble with reCAPTCHA... By using reCAPTCHA, mentioned in the article as a preferred solution, visitors from China are routinely blocked, as reCAPTCHA it is now hosted by Google. The trouble with CAPTCHA hosting: And if you intend to do anything with China based orgs.
- pfg 11y ago
  This wouldn't apply to Tor users (which this article is about), as reCAPTCHA can't identify users from China while they're using Tor.
  - TorKlingberg 11y ago
    Rather, The Great Firewall of China cannot identify reCAPTCHA (Google) when they are are using Tor. I think it blocks Tor specifically though.
    - pfg 11y ago
      Yep, that's it. I think "normal" relays get blocked almost immediately, but bridges with OBFS work to a certain degree.
deegles 11y ago
Are there any public projects that have used deep learning to defeat captchas? If not, I'm sure it's only a matter of time.
- Mahn 11y ago
  It's a cat and mouse game, of course captchas have been defeated in the past, but captchas just keep getting increasingly complex. Currently, for anonymous requests, reCAPTCHA by Google, which is what Cloudflare uses, asks you to "choose bodies of water" or street signs from a series of images, with each click sometimes revealing more options. It's fairly complex so I guess it hasn't been broken yet. It's also a massive pain in the pass for legitimate visitors.
vox_mollis 11y ago
Something people need to understand: real anonymity is really, really hard. Your COMSEC is a fairly small portion of the attack surface area, and the consequence of this is that staying anonymous is, BY NECESSITY, going to be very inconvenient. From this perspective, captchas are a very minor concern. I'm as pro-privacy as anyone, but this expectation that anonymous activity is supposed to be easy or convenient will never be satisfied. Thousands of years of lessons from both military and civilian clandestine operations bears out the critical lesson that anonymity is, by default, very very inconvenient. Nothing is going to change that.
- nxzero 11y ago
  Agree, though pointless systems that likely extract the identity of a user, force them to work for free, etc - and fail to counter the risk they supposedly stop is abusive.
CiPHPerCoder 11y ago
"We this is so important"
noonespecial 11y ago
I encountered an impossible situation working on a wordpress site the client insisted needed to be fully reachable via Tor. Parts of the page loaded from a cloudflare CDN but the main site didn't. The user was never presented with the CAPTCHA of course, 3/4 of the page was just missing with no explanation. I never did find a way around that.
- jobbleobble 11y ago
  As mentioned in the article, if you control the cloudflare CDN you can now whitelist all tor access.
  - dsl 11y ago
    I think he is talking about https://cdnjs.com/ https://cdnjs.com/ I'd really hope that CloudFlare whitelists Tor for CDNJS.
    - rdl 11y ago
      Probably not cdnjs -- there are lots of people who use CloudFlare for an assets domain (since it's free) -- if they are just serving images, it's a problem to display the CAPTCHA. It is probably best practice, if none of those assets are sensitive, to disable as much security as possible on that domain. It might be worth having some packages of defaults for tuning that. (One of the benefits for our enterprise customers is one of our staff works with them to tune settings.)
jobbleobble 11y ago
It's also worth noting that they did all this whilst getting some serious stick from some of the core Tor devs: https://trac.torproject.org/projects/tor/ticket/18361 https://trac.torproject.org/projects/tor/ticket/18361
- MichaelGG 11y ago
  I love Tor, and I like CF as a service though I'm not convinced they're a net positive for society. But wow, the non-CF guy (cypherpunks) on that ticket was really being a dick. Gotta hand it to the CF people that they kept engaging and trying to figure out some solution. Though it seems that the idea of allowing GETs when a site isn't under load attack is probably the right solution?
  - mirimir 11y ago
    All anonymous comments show up as "cypherpunks".
ownedthx 11y ago
Antecdata: most of the scamming directed towards our startup comes from Tor exit nodes.
tofupup 11y ago
I previously didn't have an opinion on cloudflare until recently using the net from a network that was black listed - web surfing was reduced to constantly filling captachas. Bottom line for me is no single organization should have so much power and I have stopped using cdn's and encourage everyone else to do the same.
hartator 11y ago
> We also made a change based on the experience of having to pass CAPTCHAs ourselves that treated all Tor exit IPs as part of a cluster, so if you passed a CAPTCHA for one you wouldn’t have to pass one again if your circuit changed. I don't get how this is different than a super cookie. Anyway, I think globally that's a well balance reaction to the TOR issue.
- schoen 11y ago
  > I don't get how this is different than a super cookie. The supercookie would survive or be detectable across multiple browser sessions (keep in mind that the Tor Browser automatically deletes regular cookies when you quit). The behavior that CloudFlare is describing here works within a single Tor Browser session but not across multiple sessions. I believe the Tor Browser is willing to send some first-party session data to a site after changing circuits, so that you wouldn't be logged out of an account if you logged in over Tor and kept that session active for long enough that Tor switched over to a different circuit. This behavior is basically what should allow CloudFlare to recognize that a particular Tor user has recently passed a CloudFlare CAPTCHA (on a particular site). However, if the user quits and restarts Tor Browser, CloudFlare will no longer be able to detect that it's the same visitor (if it could, that would be the supercookie case).
stegosaurus 11y ago
Why do CAPTCHAs even exist for standard websites (as opposed to account creation, etc)? Wouldn't something like rate limiting or proof of work achieve the same result? If you're simply allowing someone to browse, you don't actually care whether a user is real or not. You care about stopping automated comments/spam. Is this just another tentacle of the advertising industry?
- nodja 11y ago
  It's explained in the article. On the other hand, anonymity is also something that provides value to online attackers. Based on data across the CloudFlare network, 94% of requests that we see across the Tor network are per se malicious. That doesn’t mean they are visiting controversial content, but instead that they are automated requests designed to harm our customers. A large percentage of the comment spam, vulnerability scanning, ad click fraud, content scraping, and login scanning comes via the Tor network. To give you some sense, based on data from Project Honey Pot, 18% of global email spam, or approximately 6.5 trillion unwanted messages per year, begin with an automated bot harvesting email addresses via the Tor network. Rate limiting would block legitimate users, and pow doesn't impede the malicious uses of tor.
  - stegosaurus 11y ago
    That doesn't answer my question, though. All of those are reasons to suspect Tor exit nodes, but not reasons to place CAPTCHAs on standard article pages on a site. The only vaguely reasonable one I can see there is 'ad click fraud', and I think that fundamentally restricting the usefulness of a site for advertising purposes is awful.
    - deleted 11y ago
      [deleted]
hueving 11y ago
A "whitelist" tor solution that isn't whitelisting tor by default is really lame. Approximately 0% of website operators will think to enable that so tor users are generally still treated like garbage visiting any CF protected website.
cyphar 11y ago
> If we provide a way to treat Tor differently by applying a rule to whitelist the network's IPs we couldn't think of a justifiable reason to not also provide a way to blacklist the network as well. Yes there is: "we don't provide censorship as a service".
foxylad 11y ago
I'm one of Cloudflare's customers who would blacklist all Tor traffic if I could. I genuinely don't understand why so many people obviously use Tor for all their browsing, and not just for sites where remaining anonymous is desirable. Why not simply switch to a normal browser for normal sites? Some background - we run several SaaS services for schools, which are politically and socially non-sensitive. The only realistic reasons anyone would want to connect anonymously would be nefarious. Allowing Tor traffic is like a bank having a special ATM round the back with no security cameras - you're giving a free pass to attackers to try anything they want with impunity. I'm having a hard time seeing what the compensating advantage is. How does not accepting Tor traffic to our "normal" sites lessen the anonymity of Tor traffic to sites where it is important?
- voltagex_ 11y ago
  The canonical answer is https://www.torproject.org/about/torusers.html.en https://www.torproject.org/about/torusers.html.en Really, no one has any business looking at anything I do on the Internet. I don't use Tor for everything but I may use it when I'm connecting via a network that I believe to be hostile (i.e. just about everything outside of my house)
  - ashitlerferad 11y ago
    > (i.e. just about everything outside of my house) Which ISP do you use? I would be surprised if a trustworthy ISP exists.
    - voltagex_ 11y ago
      Internode, in Australia. They're subject to the data retention laws, so I use HTTPS and VPNs as necessary. When they're completely consumed by their new owners, TPG, then no, there won't be a trustworthy ISP in Australia. I have high hopes for SkyMesh, though.
- fabulist 11y ago
  Then allow the Tor users and watch them carefully. If you think you have a way to definitively tell that some traffic is malicious, why oh why would you interrupt your enemy while they're making a mistake? Do you think they're not going to come back at you from a clearnet IP? Edit: > I'm having a hard time seeing what the compensating advantage is. How does not accepting Tor traffic to our "normal" sites lessen the anonymity of Tor traffic to sites where it is important? The anonymity of Tor depends upon diversity. The more people using Tor for more things, the harder it is to correlate any particular person's traffic. That being said, it is your website, and if you decide to block Tor you have as much right as your users do to use Tor. But I'd ask you to think about whether you are actually attaining any benefit.
dredmorbius 11y ago
I'm glad to see CloudFlare addressing Tor users and issues with CAPTCHA, as I've been victim of this myself multiple times in recent years. In particular is the issue that CloudFlare assumes javascript-enabled browsers, a condition which may well not be met. I recorded an exchange with CloudFlare support some time back in which the CloudFlare rep was apparently unaware how or why this might occur: https://plus.google.com/104092656004159577193/posts/H2sakaRxMPA https://plus.google.com/104092656004159577193/posts/H2sakaRx... I'm also aware of some tools/approaches which address the question of fair anonymity -- ensuring well-behaved clients while retaining anonymous status for the client. Best I'm aware these are very experimental. I've also forwarded the information to TK Hyponnen of F Secure, who may have some impression of the approaches. FAUST: https://gnunet.org/node/1704 https://gnunet.org/node/1704 (Efficient, TTP-Free Abuse Prevention by Anonymous Whitelisting | GNUnet) Fair Anonymity: http://arxiv.org/pdf/1412.4707v1.pdf http://arxiv.org/pdf/1412.4707v1.pdf Assessing these is beyond my skills, but the references may be useful to CloudFare (or others).
MaddoScientisto 11y ago
The one time I used TOR the captchas were the ones with both words as squiggly letters which wouldn't work even after 20 attempts, I had to give up. I was pleased when, recently, I found out they switched to the image based one. Sure, sometimes it still refuses to accept that I selected all the street signs but at least I don't have to give up in frustration after 30 consecutive failed attempts

323 points by jgrahamc 11y ago

Probably not really suitable for Cloudflare scale, but for others who are forced to use CAPTCHA, consider https://hashcash.io/ https://hashcash.io/ for easy Proof-of-Work integration.

deleted 11y ago

[deleted]

sajid 11y ago

Great blog. Some of the best technical writing out there.

nxzero 11y ago

"Trouble with Cloudflare" Treating TOR traffic the same as non-TOR traffic makes no sense; read the main link for confirmation they do. Case in point, and for starters, STOP repeatively requiring a user from a session to keep passing "I'm not a robot" tests. Set a global cookie that's valid for the session, across all of Cloudflair's network, and honor it. If the "I'm not a robot test" doesn't work unless it's repeatively given, then that is the problem, not TOR. Please address this issue; thanks.

x1798DE 11y ago
That wouldn't really help much, because then a bot owner just needs to solve one CAPTCHA, receive the "I'm not a robot" cookie, then hand it over to their bot.
- nxzero 11y ago
  Right, which is the issue, CAPTCHA's don't work.
  - noxToken 11y ago
    Disagree. They don't work if you only show one CAPTCHA. Your proposal defeats the entire purpose, because it only takes human interaction once to defeat CAPTCHA over CloudFlair's entire network. CAPTCHA isn't a silver bullet, and I don't think anyone is lauding it as the end-all be-all to stop spammers and malicious activity.
    - nxzero 11y ago
      Have you used TOR and experienced what Cloudflair does? The server 3-10 tests per page requested in some instances collecting massive amount to training data; I'd in fact be suprise do if Google doesn't place an incentive on collecting as much data as possible; likely enough to ID a user 99% of the time based on the input provided by the user. As for it not being a silver bullet, it's well known that CAPTCHA don't stop bots, but mine data from people. There's not a single Google "I'm not a robot" that can't easily be circumvented, I just know most TOR users aren't able to and Google & Cloadflair are exploiting this.
alex_duf 11y ago
If you set a global cookie, then someone intercepting the public traffic (think NSA, GCHQ,...) can identify what the user is reading. You just killed anonymity.
- nxzero 11y ago
  Global cookie is a session, if the user wipes the cookie, resets the TOR connection, etc. that's their issue. TOR is not designed to hide sessions, nor would setting a global cookie break anonymity unless the user doesn't understand how TOR works. All exit nodes are watched and session device fingerprints are correlated with or without a global cookie.
  - ikeboy 11y ago
    If nobody set a global cookie, a passive attacker cannot correlate different tabs using different circuits. See https://trac.torproject.org/projects/tor/ticket/3455 https://trac.torproject.org/projects/tor/ticket/3455 and https://www.torproject.org/projects/torbrowser/design/#identifier-linkability https://www.torproject.org/projects/torbrowser/design/#ident...
    - nxzero 11y ago
      If the same global cookie is accessible via two circuits, that a bug in a product that uses TOR, not TOR; I personally go above and beyond simply creating a new circuit, never open two circuits at the same time or boot, limit TOR sessions to single use, and locally compartmentize data per session, etc. TOR is not plug in play, it takes effort and discipline, and will never be a fully automated solution.
      - ikeboy 11y ago
        Yes, TBB on default settings is vulnerable to associating multiple tabs (if I'm reading the link above right), if an adversary sets a shared cookie. That does not mean it's ok for someone to set a shared cookie. The possibility of exploitation does not mean exploitation or making exploitation easier is fine.

amelius 11y ago

CAPTCHAs are essentially a broken idea: it is easy for an attacker to send the CAPTCHA image to another website and ask users to fulfill the CAPTCHA for a completely unrelated goal. This trick has been used in the past on certain pr0n websites (users are allowed to see a picture only after they complete the CAPTCHA). Also, one could use a mechanical turk service to circumvent CAPTCHAs.

basch 11y ago
reCAPTCHA does things like track your mouse movement and a bunch of other hocus pocus.
- amelius 11y ago
  > and a bunch of other hocus pocus Sounds like security by obscurity. By the way, some people use a track pad (with a stylus), where the mouse can jump discontinuously.
  - Klathmon 11y ago
    They aren't taking any one thing at face value, but are combining them to get a better picture. You might use a trackpad, so you'd "fail" that test, but your useragent is normal, you've been seen before with those cookies, and your IP is good so you are fine. But if your IP is a known "bad actor", your useragent is something never before seen, your mouse movements are abnormal, and your keyboard inputs are instant, well all of that combined means you are getting blocked.
    - amelius 11y ago
      > But if your IP is a known "bad actor", your useragent is something never before seen, What if I install a new computer on an IP address freshly provided to me by my ISP? Or what if I just open a new incognito window? Will I get blocked? > your mouse movements are abnormal, and your keyboard inputs are instant It seems to me these are really easy to fake programmatically.

lazyjones 11y ago

What's the rationale for this weak crypto in Tor? Tor uses hashes generated with the weak SHA-1 algorithm to generate .onion addresses and then only uses 80 bits of the 160 bits from the hash to generate the address, making them even weaker. Other weaknesses include use of RSA-1024, people have been complaining to no avail since at least 2013: http://arstechnica.com/security/2013/09/majority-of-tor-crypto-keys-could-be-broken-by-nsa-researcher-says/ http://arstechnica.com/security/2013/09/majority-of-tor-cryp...

petertodd 11y ago
This isn't actually a vulnerability yet, as SHA-1 is only known to be vulnerable to collision attacks (where you try to find two messages with the same hash) rather than pre-image attacks (where you try to find a message with a specific hash); almost no hash functions have ever been found to be vulnerable to pre-image attacks: https://github.com/zooko/hash-function-survey/blob/master/preimage-attacks-color.rst https://github.com/zooko/hash-function-survey/blob/master/pr... Secondly, the issue is moot because generating collisions on 80 bits takes just 2^40 work - not hard at all. Basically what that means for Tor, is that while it'd be pretty easy for a Onion site operator to generate two keys corresponding to the same .onion address, an _attacker_ still has to do 2^80 work to attack a site by generating a key with the same Onion address. While that's not great - 2^128 work is considered "standard" in cryptographic work - 2^80 work is still hard enough that there are probably cheaper ways of attacking Onion sites (for reference, the cumulative total work done by all Bitcoin network miners in the entire history of Bitcoin is about 2^80 hashes). As for the 1024bit pubkeys, I'm not sure what the status of that is; from what I hear Tor is actively working towards a Onion redesign that will fix these issues, and longer pubkeys may have already been fixed.
- zmanian 11y ago
  Tor 224 proposes moving hidden service addresses to full ed25519 keys rather than truncated hashes of RSA keys amongst other things. https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.txt https://gitweb.torproject.org/torspec.git/tree/proposals/224...
- 11y ago

Klathmon 11y ago

"To make sure our team understood what a pain CAPTCHAs could be, I blacklisted all the IP addresses used in CloudFlare's office so our employees would need to pass a CAPTCHA every time they wanted to visit any of our customers' sites." Say what you will about cloudflare, that's an impressive move.

jgrahamc 11y ago
What he didn't say was that he did it without warning and it lasted 30 days. Imagine what that was like for the technical support and customer success teams who were helping customers with their sites.
- deleted 11y ago
  [deleted]
- brianwawok 11y ago
  I am a strong believer in dogfooding. If it wasn't a pain you felt, it wouldn't be prioritized to be fixed.
  - sievebrain 11y ago
    Dogfooding is great, if there actually is a fix to the issue at hand. If the issue is "anonymous internet access gets abused a lot" then I'm not sure what a tech support guy is meant to usefully contribute to ending the dogfooding pain.
    - throwaway7767 11y ago
      > Dogfooding is great, if there actually is a fix to the issue at hand. If the issue is "anonymous internet access gets abused a lot" then I'm not sure what a tech support guy is meant to usefully contribute to ending the dogfooding pain. The problem being solved was not "anonymous internet access gets abused a lot", it was "the mechanism we use to combat abuse is too aggressive for some segments of our users and thereby denying service to actual humans, and these challenges are invisibile to our engineers and employees because they don't browse on connections that trigger the system". As the blog post shows (and this is backed up by my experience as a user of Tor) CloudFlare significantly improved its handling of this in response. So I would say this is a success. I am sure that pressure from employees, including support guys, caused this long-running issue to finally get the attention it deserved.

belorn 11y ago

One should always be doubting when it comes to security features that do not present the benefits in a clear way. The article for example claims that 18% of global email spam comes from harvested emails that is collected using tor, but is those 18% exclusively using tor? Do uses who publisher their email address on their website that is hosted through cloudfare see a lower number of spam? It should be a fairly easy thing for cloudfare to test, while also testing vulnerability and login attempts. As an aside, it would also be interesting to see if there is a quality vs quantitative differences in the malicious activity (ie, if serious attempts are done through botnets, and script kiddie activity is done through tor). The last a final test in order to verify a security measure that has such a high cost as this one, is to ask if its has any meaningful impact to the end result. A website with 10000 vulnerability scans per day is not going to be meaningful improved if it was reduced to 5000 per day, even if that is a 50% reduction. If there is a known vulnerability, the site is going to get hijacked either way.

wolfgke 11y ago

This is why people should browser a lot more over Tor, such that the relative amount of malicious traffic is reduced and thus operators will not be able to uphold the argument that Tor is often malicious traffic anymore.

seabee 11y ago
Unfortunately, it won't work. 1. It's a charity tax; you have to convince people to incur the cost of Tor (i.e. CAPTCHAs everywhere) for activities that don't require Tor. 2. You can't neutralise a poison by diluting it. Firstly, from the operators' POV, if there's a widespread agreement that people use Tor even though they don't need to, then they know voluntary users can be pressured not to use Tor through sheer inconvenience. Even if you wanted to boycott a service that blocked Tor, it's notoriously hard to make good on that threat unless you wield a lot of power or annoyed a very large number of people. So the consequences are minor. Secondly, the percentage of malicious Tor traffic is a red herring. What operators care about is the origins of malicious traffic. If 50% of your attacks come from one particular country (or Tor) and the cost of losing that traffic is less than the cost of that malicious traffic, there is a real incentive to block that traffic. Combined with the first point, the cost of losing voluntary Tor users is insignificant if they can easily choose not to use it.
- wolfgke 11y ago
  > 1. It's a charity tax; you have to convince people to incur the cost of Tor (i.e. CAPTCHAs everywhere) for activities that don't require Tor. People are willing to invest personal ressources for charitable purposes. Why not here? People are willing to fight against discrimination. Why not against discrimination of Tor users? > 2. You can't neutralise a poison by diluting it. There is also poisonous traffic from non-Tor adresses. > Combined with the first point, the cost of losing voluntary Tor users is insignificant if they can easily choose not to use it. People would strictly avoid restaurants that don't serve coloured people. Why don't they avoid services that don't serve Tor users?
  - Vendan 11y ago

lake99 11y ago

HN is the only place I bother solving CAPTCHAs. For everything else, Firefox (Tor Browser) has plugins to get a copy from arhive.org, archive.is, or google-cache. So if the page asks me to solve a CAPTCHA, I don't visit them. This could be a benefit for the website (lower server load) or a harm (fewer people appear to be reading their content, fewer people see their ads). Whatever the case may be, I'm caught in the crossfire between crackers and servers. I don't care about their war at all. As far as I'm concerned, I'm winning. Cloudfare wanted me to solve a CAPTCHA to read their article. I tried to archive it, but arhive.is already had a copy of it. This happens to me quite often. So, obviously, I'm not the only one who has figured out a way around their war. > Security, Anonymity, Convenience: Pick Any Two Nah, I usually have all three.

scrollaway 11y ago
> I usually have all three ... > Firefox (Tor Browser) has plugins to get a copy from arhive.org, archive.is, or google-cache. So if the page asks me to solve a CAPTCHA, I don't visit them. You don't have convenience.
- Sir_Cmpwn 11y ago
  A one-click measure to circumvent a captcha is pretty good convenience-wise imo.
  - scrollaway 11y ago
    Sure, but google cache/archive pages often lack some images, have broken javascript, etc. Additionally, there's a huge difference between "point and click", and "point, click, click again, look for the plugin, another click, do this for every page".
    - lake99 11y ago
      > "point, click, click again, look for the plugin, another click, do this for every page". Honestly, I don't know how I can make the process as complicated as you described it, even if I wanted to. In reality, it is no more complicated than right-click, open in new tab.

rnhmjoj 11y ago

Why is everyone obsessed with knowing whether the user is a bot or an actual person? What differences does it make? A bot it's not inherently malicious, there are thousand of legitimate use cases: a bot may be downloading content as part of a script or some application, checking updates or creating a cache. Traffic generated by bots should not be blocked per se. There are certainly lots of malicious bots scanning for vulnerabilities, DDoSing sites and so on but this applies to people as well. I think human generated traffic may have priority but blocking bots entirely is nonsense: ultimately the user agent is always a "bot" acting on behalf of an actual person: by blocking this traffic you may always break some user workflow.

jeffasinger 11y ago
CAPTCHAs are useful when you want to rate limit something to an extremely low rate, like for example attempting to login with a username and password.
- rnhmjoj 11y ago
  You could slow down the responses or rate limit the requests, with no need to completely block automatic logins.
  - progval 11y ago
    You would have to limit it per IP address if you do not want to after all clients. So it would not be effective against someone who can use many IP addresses (eg. with Tor)
  - jeffasinger 11y ago
    If you are a valuable enough target, this is not an option. IPs are cheap, if you let someone try 20 times in an hour before banning an IP, there are targets that people will cycle through IPs that quickly for.
  - dredmorbius 11y ago
    Tor largely prevents most approaches to this. You'd need some way to provide a ticket to a given client, check for it later, and, preferably, ensure anonymity over time. I've just posted top-level in this thread listing two projects of which I'm aware that provide this, though as experimental protocols only. I've been mildly agitating for further development of such tools. Looks as if CloudFlare are working in a similar direction, which I see as positive.

monoclonal 11y ago

Worth noting that a surprisingly common amount of sites have other sections of their site which are not routed through Cloudflare. Look for instances of DNS records like this: admin.example.com Such a record is usually not routed through Cloudflare because the last thing a webmaster wants is to solve captchas for their own website. They don't however care much for their visitors if they're subjecting (possibly a substantial amount of them) to captcha solving nonsense. The content in the non-cf sections of a site can still be accessed because the webmaster is lazy and didn't care to check if a visitor can do a DNS DIG on all their DNS records. Or you can simply use TOR pluggable transports to pretend you're Googlebot, and also hide all your traffic in Google-like traffic. I would reserve this for rare cases as there are people in censorship prone countries who really need this bandwidth :)

besselheim 11y ago
The pluggable transports are for connections from your Tor client into the Tor network, not connections from exit nodes to the rest of the Internet. Cloudflare (or any destination host) would still be able to detect your connection as originating via Tor.
- monoclonal 11y ago
  There are innumerable ways infact to spoof the fact you're not using TOR to a website, and you can read up on these in the TOR documentation. Ideally you're looking to use TOR as the first hop, and then you dial into the wider Internet with a VPN, or as I mentioned: Using various Google services to camouflage traffic instead of a VPN. This is where pluggable transports come in, because Google doesn't like TOR, so you want to choose how you're connecting to Google, and get to traverse the TOR network to find an optimal route.
  - besselheim 11y ago
    Pluggable transports are intended to stop your local ISP detecting or blocking your connection to Tor: https://gitweb.torproject.org/torspec.git/tree/pt-spec.txt https://gitweb.torproject.org/torspec.git/tree/pt-spec.txt While I agree that you can mask your exit from the Tor network via an additional proxy or VPN, that's not the role of pluggable transports. They're only for connecting in to the Tor network, not out from it.

thraway235 11y ago

anyone else concerned that the captcha's offer an ability to de-anonymize someone completely? for example having a backend algo offer certain captcha's that show up only in certain areas of the world? I feel like this is entirely possible and is part of the reason I will not complete and captcha's moving forward.

jnagro 11y ago
definitely a big problem. and its Google who has all the data.
TorKlingberg 11y ago
When using Tor, reCAPTCHA cannot tell where you are.

barrkel 11y ago

I have never successfully completed a captcha served up by cloudflare (and thus Google) on Tor. They are fiendishly difficult to the point I suspected the mechanism is broken.

Sir_Cmpwn 11y ago
+1 I just give up immediately when I see a Cloudflare captcha page on Tor.
throwaway7767 11y ago
> I have never successfully completed a captcha served up by cloudflare (and thus Google) on Tor. They are fiendishly difficult to the point I suspected the mechanism is broken. Have you tried recently? They have gotten way better. There was a time not long ago where I would have agreed with you (the CAPTCHAs were literally impossible for a human in most cases). CloudFlare has toned the CAPTCHAs down a lot recently, they're now presenting image classification tasks (select street signs/bodies of water/storefronts/cactuses/...), and in my experience, my success rate is close to 100% on those. Admittedly they still make me solve three of them and they get tiring, but at least they're not completely cutting off access to Tor users anymore.
buro9 11y ago
I use the audio captcha. Works every time. The ones that are impossible for my are the street signs, and after that all of the cultural ones (I see US captchas and when asked to select a sandwich or a recreational vehicle I'm doomed to not complete them). An interesting side effect of failing a captcha is that to Google this looks like proof the captcha is working, that you're likely to be a bot, and that they should definitely give you the hard captchas. As such, if you cannot complete a captcha the chances increase that you must now complete multiple difficult captchas. The audio captcha is delightfully simpler though it does take a moment longer to complete.
- MaddoScientisto 11y ago
  Doesn't the audio captcha work only if you have javascript enabled? If you enable javascript on TOR then you are doing it wrong. I could be wrong though

tech-no-logical 11y ago

I don't use Tor daily (although I run a small exit node), but I do surf via a VPN. cloudflare's captchas drive me to the brink of insanity, I see at least 50 each day...

brianwawok 11y ago
Have you considered setting up your own VPN? That would make the ip "clean" and get less captchas, right?
- amdavidson 11y ago
  > Have you considered setting up your own VPN? That would make the ip "clean" and get less captchas, right? And would negate any anonymity offered by using a VPN.
  - mathrawka 11y ago
    Tor : Anonymity :: VPN : Privacy VPNs do not provide anonymity.
    - startling 11y ago
      I think you misunderstood -- I parsed it as "And using a VPN would negate any anonymity offered (by using Tor)".
Zarel 11y ago
Cloudflare says: > With most browsers, we can use the reputation of the browser from other requests it’s made across our network to override the bad reputation of the IP address connecting to our network. For instance, if you visit a coffee shop that is only used by hackers, the IP of the coffee shop's WiFi may have a bad reputation. But, if we've seen your browser behave elsewhere on the Internet acting like a regular web surfer and not a hacker, then we can use your browser’s good reputation to override the bad reputation of the hacker coffee shop's IP. I occasionally use a VPN, but I've never gotten a CloudFlare captcha. Is it possible that you might be doing something else other than just using a VPN, such as blocking cookies?

fosco 11y ago

captcha's are evil. unfortunately I do not trust tor.[0][1] I'm unsure the internet will ever be anonymouse unless large completely private networks start gaining popularity. [0]http://fusion.net/story/238742/tor-carnegie-mellon-attack/ http://fusion.net/story/238742/tor-carnegie-mellon-attack/ (11/2015) [1]http://www.theguardian.com/technology/2014/jul/22/is-tor-truly-anonymising-conference-cancelled(7/2014) http://www.theguardian.com/technology/2014/jul/22/is-tor-tru...

beeboop 11y ago

In addition to CAPTCHAs, why not just have a button that runs some JavaScript that completes a proof-of-work similar to what mining bitcoins does? You could make it only take 5 seconds on a modern laptop CPU, about as long as it'd take to enter the CAPTCHA anyway, but it'd potentially be a very large road block for spammers/DDOSers. For those on phones, you can still opt for CAPTCHA if you don't want to kill your battery.

JoeAltmaier 11y ago
Love it! I hope almost all authentication will become automated. My personal client machine is much better at proving who it (I) am that a human.
mikeash 11y ago
I did this with the comments box on my blog. I don't think it's technically effective, because doing proof of work in JavaScript is orders of magnitude slower than doing it natively (especially with GPU acceleration). It works well enough for me, because I'm not a big enough target, but it wouldn't be a major obstacle for someone who really wanted to cause havoc. It would be interesting to see how fast you could make the JavaScript code. I'm sure my version is just terribly unoptimized. The requirement to support the least common denominator will present a major problem, though.
- tne 11y ago
  I would look at asm.js, do some tests with major browsers and beef it up. It would make it painful under user agents that don't JIT (or compile AOT) by leveraging asm.js conventions (many do though). If the wasm effort works out (it's looking like it will), it would hopefully alleviate the issue you present entirely and make this solution viable in a very sane way.
pfg 11y ago
Spending 5 CPU seconds to submit one comment might be significantly cheaper for a spammer than paying someone to solve the captcha for them. Additionally, specifically with Tor users, you can expect a large chunk of the user base to have JavaScript disabled completely. You can do many things with JavaScript that could be used to build a browser fingerprint, so someone who's already using software to browse the web anonymously is very likely to disable that.

mdip 11y ago

I played around with Tor a couple of months ago when it hit the headlines again, just to see what it was all about, and the experience was awful. As an exercise, I decided to fire up Tor browser today just to see if it's gotten any better. It's worse. Here's what I found: Large Image based CAPTCHA: Tor is slow. I'm on a 75/25Mbps internet connection and it loads images similar to 24k dial-up. The CAPTCHA I was presented with was the highest bandwidth CAPTCHA I've ever seen. I was given a 9 pictures and needed to select "Bodies of Water". Each click yielded a new square. I had to wait for 4 additional images to load before I could click "Validate". This took over a minute. Then, repeat, this time "Store Fronts", which were hard to discern (is it a blurry Apartment Building front or Store Front?). I received a connection error on one site so I had to repeat the process. With Javascript features turned off, it was a little easier, but included the extra step of having to paste a Base64 encoded string into a text box which failed twice. Every site I tried gave me this CloudFlare CAPTCHA page. One of the sites I pulled up had no images. I set my privacy settings to the least protected, enabling JavaScript and HTML5, assuming this was the problem. Nope. They used images from another site and I had to grab the image URL and paste it into a browser to see what was going on. It was yet another CAPTCHA. A few minutes later, the previous site displayed images properly. On to Google. Privacy settings are still at the weakest setting. Type "Google" into the search engine and I get a "wavy text in an image" CAPTCHA. This loads quickly and is easy to answer, but just results in another CAPTCHA. I gave up after 10 tries. Bing, Yandex and Yahoo all worked with Yandex only presenting a CAPTCHA once after the third search I did (simple, like Google, but worked). This is a terrible experience for people seeking to get around oppressive governments. While I applaud CloudFlare for dogfooding their CAPTCHA system, I doubt they did it in a way that truly simulates the experience via an extraordinarily slow internet connection which is what I ended up with when using Tor Browser. I wonder how much slower things would be if my internet connection was 1Mbps or being interfered with by government infrastructure. I understand the trade-off between securing a site from "evil traffic" that is more likely to originate from a Tor exit node, but why must they use such a bandwidth intensive CAPTCHA? A browsing experience that would have taken seconds to complete took me almost 5 minutes (and a lot of frustration) not including actually consuming the content I was looking for. Are the text in image based CAPTCHAs not good enough for this task? Are there other reasons I'm missing?

dev1n 11y ago

For users who can afford it, routing a vpn through tor solves the captcha issue with cloudflare. Also adds an extra layer of security

amdavidson 11y ago
Can you elaborate? I cannot imagine how Cloudflare could distinguish VPN traffic routed through Tor and standard traffic routed through Tor. The only difference is a hop on the front end, no change to what comes out the exit node.
- dev1n 11y ago
  if you setup an access point to route all of your traffic through tor, then connect to your VPN through that access point, your IP is the VPN IP, not the exit node.
  - icebraining 11y ago
    What the point of doing that over connecting directly to the VPN? Seems like the benefit granted by Tor (avoiding leaking who connected to the VPN) would be negated by the fact that there are now payment records from you to the VPN.
    - dev1n 11y ago
      If you pay with BTC, prepaid gift card (paid for in cash) etc.. then there are no payment records from you to the VPN. The benefit of this is that the VPN provider doesn't know who you are because you are accessing the VPN through tor. Yet the VPN provides you a stable IP that won't be CAPTCHA'd like most normal tor exit nodes are. Edit: This is good if you are trying to maintain an Internet profile (i.e. Facebook, twitter etc.) that isn't tied to your true identity.
      - jobbleobble 11y ago
        But you are losing out on some anonymity here. The VPN provider may not know who you are but you are consistently making access with the same user ID and from the same IP. Your activity can be correlated to that account and that IP. If that's not your aim (like you say - being signed in to the same facebook account all day suffers you the same problem) then this isn't an issue. But this isn't what a lot of tor users want tor for.

zhte415 11y ago

My trouble with reCAPTCHA... By using reCAPTCHA, mentioned in the article as a preferred solution, visitors from China are routinely blocked, as reCAPTCHA it is now hosted by Google. The trouble with CAPTCHA hosting: And if you intend to do anything with China based orgs.

pfg 11y ago
This wouldn't apply to Tor users (which this article is about), as reCAPTCHA can't identify users from China while they're using Tor.
- TorKlingberg 11y ago
  Rather, The Great Firewall of China cannot identify reCAPTCHA (Google) when they are are using Tor. I think it blocks Tor specifically though.
  - pfg 11y ago
    Yep, that's it. I think "normal" relays get blocked almost immediately, but bridges with OBFS work to a certain degree.

deegles 11y ago

Are there any public projects that have used deep learning to defeat captchas? If not, I'm sure it's only a matter of time.

Mahn 11y ago
It's a cat and mouse game, of course captchas have been defeated in the past, but captchas just keep getting increasingly complex. Currently, for anonymous requests, reCAPTCHA by Google, which is what Cloudflare uses, asks you to "choose bodies of water" or street signs from a series of images, with each click sometimes revealing more options. It's fairly complex so I guess it hasn't been broken yet. It's also a massive pain in the pass for legitimate visitors.

vox_mollis 11y ago

Something people need to understand: real anonymity is really, really hard. Your COMSEC is a fairly small portion of the attack surface area, and the consequence of this is that staying anonymous is, BY NECESSITY, going to be very inconvenient. From this perspective, captchas are a very minor concern. I'm as pro-privacy as anyone, but this expectation that anonymous activity is supposed to be easy or convenient will never be satisfied. Thousands of years of lessons from both military and civilian clandestine operations bears out the critical lesson that anonymity is, by default, very very inconvenient. Nothing is going to change that.

nxzero 11y ago
Agree, though pointless systems that likely extract the identity of a user, force them to work for free, etc - and fail to counter the risk they supposedly stop is abusive.

CiPHPerCoder 11y ago

"We this is so important"

noonespecial 11y ago

I encountered an impossible situation working on a wordpress site the client insisted needed to be fully reachable via Tor. Parts of the page loaded from a cloudflare CDN but the main site didn't. The user was never presented with the CAPTCHA of course, 3/4 of the page was just missing with no explanation. I never did find a way around that.

jobbleobble 11y ago
As mentioned in the article, if you control the cloudflare CDN you can now whitelist all tor access.
- dsl 11y ago
  I think he is talking about https://cdnjs.com/ https://cdnjs.com/ I'd really hope that CloudFlare whitelists Tor for CDNJS.
  - rdl 11y ago
    Probably not cdnjs -- there are lots of people who use CloudFlare for an assets domain (since it's free) -- if they are just serving images, it's a problem to display the CAPTCHA. It is probably best practice, if none of those assets are sensitive, to disable as much security as possible on that domain. It might be worth having some packages of defaults for tuning that. (One of the benefits for our enterprise customers is one of our staff works with them to tune settings.)

jobbleobble 11y ago

It's also worth noting that they did all this whilst getting some serious stick from some of the core Tor devs: https://trac.torproject.org/projects/tor/ticket/18361 https://trac.torproject.org/projects/tor/ticket/18361

MichaelGG 11y ago
I love Tor, and I like CF as a service though I'm not convinced they're a net positive for society. But wow, the non-CF guy (cypherpunks) on that ticket was really being a dick. Gotta hand it to the CF people that they kept engaging and trying to figure out some solution. Though it seems that the idea of allowing GETs when a site isn't under load attack is probably the right solution?
- mirimir 11y ago
  All anonymous comments show up as "cypherpunks".

ownedthx 11y ago

Antecdata: most of the scamming directed towards our startup comes from Tor exit nodes.

tofupup 11y ago

I previously didn't have an opinion on cloudflare until recently using the net from a network that was black listed - web surfing was reduced to constantly filling captachas. Bottom line for me is no single organization should have so much power and I have stopped using cdn's and encourage everyone else to do the same.

hartator 11y ago

> We also made a change based on the experience of having to pass CAPTCHAs ourselves that treated all Tor exit IPs as part of a cluster, so if you passed a CAPTCHA for one you wouldn’t have to pass one again if your circuit changed. I don't get how this is different than a super cookie. Anyway, I think globally that's a well balance reaction to the TOR issue.

schoen 11y ago
> I don't get how this is different than a super cookie. The supercookie would survive or be detectable across multiple browser sessions (keep in mind that the Tor Browser automatically deletes regular cookies when you quit). The behavior that CloudFlare is describing here works within a single Tor Browser session but not across multiple sessions. I believe the Tor Browser is willing to send some first-party session data to a site after changing circuits, so that you wouldn't be logged out of an account if you logged in over Tor and kept that session active for long enough that Tor switched over to a different circuit. This behavior is basically what should allow CloudFlare to recognize that a particular Tor user has recently passed a CloudFlare CAPTCHA (on a particular site). However, if the user quits and restarts Tor Browser, CloudFlare will no longer be able to detect that it's the same visitor (if it could, that would be the supercookie case).

stegosaurus 11y ago

Why do CAPTCHAs even exist for standard websites (as opposed to account creation, etc)? Wouldn't something like rate limiting or proof of work achieve the same result? If you're simply allowing someone to browse, you don't actually care whether a user is real or not. You care about stopping automated comments/spam. Is this just another tentacle of the advertising industry?

nodja 11y ago
It's explained in the article. On the other hand, anonymity is also something that provides value to online attackers. Based on data across the CloudFlare network, 94% of requests that we see across the Tor network are per se malicious. That doesn’t mean they are visiting controversial content, but instead that they are automated requests designed to harm our customers. A large percentage of the comment spam, vulnerability scanning, ad click fraud, content scraping, and login scanning comes via the Tor network. To give you some sense, based on data from Project Honey Pot, 18% of global email spam, or approximately 6.5 trillion unwanted messages per year, begin with an automated bot harvesting email addresses via the Tor network. Rate limiting would block legitimate users, and pow doesn't impede the malicious uses of tor.
- stegosaurus 11y ago
  That doesn't answer my question, though. All of those are reasons to suspect Tor exit nodes, but not reasons to place CAPTCHAs on standard article pages on a site. The only vaguely reasonable one I can see there is 'ad click fraud', and I think that fundamentally restricting the usefulness of a site for advertising purposes is awful.
  - deleted 11y ago
    [deleted]

hueving 11y ago

A "whitelist" tor solution that isn't whitelisting tor by default is really lame. Approximately 0% of website operators will think to enable that so tor users are generally still treated like garbage visiting any CF protected website.

cyphar 11y ago

> If we provide a way to treat Tor differently by applying a rule to whitelist the network's IPs we couldn't think of a justifiable reason to not also provide a way to blacklist the network as well. Yes there is: "we don't provide censorship as a service".

foxylad 11y ago

I'm one of Cloudflare's customers who would blacklist all Tor traffic if I could. I genuinely don't understand why so many people obviously use Tor for all their browsing, and not just for sites where remaining anonymous is desirable. Why not simply switch to a normal browser for normal sites? Some background - we run several SaaS services for schools, which are politically and socially non-sensitive. The only realistic reasons anyone would want to connect anonymously would be nefarious. Allowing Tor traffic is like a bank having a special ATM round the back with no security cameras - you're giving a free pass to attackers to try anything they want with impunity. I'm having a hard time seeing what the compensating advantage is. How does not accepting Tor traffic to our "normal" sites lessen the anonymity of Tor traffic to sites where it is important?

voltagex_ 11y ago
The canonical answer is https://www.torproject.org/about/torusers.html.en https://www.torproject.org/about/torusers.html.en Really, no one has any business looking at anything I do on the Internet. I don't use Tor for everything but I may use it when I'm connecting via a network that I believe to be hostile (i.e. just about everything outside of my house)
- ashitlerferad 11y ago
  > (i.e. just about everything outside of my house) Which ISP do you use? I would be surprised if a trustworthy ISP exists.
  - voltagex_ 11y ago
    Internode, in Australia. They're subject to the data retention laws, so I use HTTPS and VPNs as necessary. When they're completely consumed by their new owners, TPG, then no, there won't be a trustworthy ISP in Australia. I have high hopes for SkyMesh, though.
fabulist 11y ago
Then allow the Tor users and watch them carefully. If you think you have a way to definitively tell that some traffic is malicious, why oh why would you interrupt your enemy while they're making a mistake? Do you think they're not going to come back at you from a clearnet IP? Edit: > I'm having a hard time seeing what the compensating advantage is. How does not accepting Tor traffic to our "normal" sites lessen the anonymity of Tor traffic to sites where it is important? The anonymity of Tor depends upon diversity. The more people using Tor for more things, the harder it is to correlate any particular person's traffic. That being said, it is your website, and if you decide to block Tor you have as much right as your users do to use Tor. But I'd ask you to think about whether you are actually attaining any benefit.

dredmorbius 11y ago

I'm glad to see CloudFlare addressing Tor users and issues with CAPTCHA, as I've been victim of this myself multiple times in recent years. In particular is the issue that CloudFlare assumes javascript-enabled browsers, a condition which may well not be met. I recorded an exchange with CloudFlare support some time back in which the CloudFlare rep was apparently unaware how or why this might occur: https://plus.google.com/104092656004159577193/posts/H2sakaRxMPA https://plus.google.com/104092656004159577193/posts/H2sakaRx... I'm also aware of some tools/approaches which address the question of fair anonymity -- ensuring well-behaved clients while retaining anonymous status for the client. Best I'm aware these are very experimental. I've also forwarded the information to TK Hyponnen of F Secure, who may have some impression of the approaches. FAUST: https://gnunet.org/node/1704 https://gnunet.org/node/1704 (Efficient, TTP-Free Abuse Prevention by Anonymous Whitelisting | GNUnet) Fair Anonymity: http://arxiv.org/pdf/1412.4707v1.pdf http://arxiv.org/pdf/1412.4707v1.pdf Assessing these is beyond my skills, but the references may be useful to CloudFare (or others).

MaddoScientisto 11y ago

The one time I used TOR the captchas were the ones with both words as squiggly letters which wouldn't work even after 20 attempts, I had to give up. I was pleased when, recently, I found out they switched to the image based one. Sure, sometimes it still refuses to accept that I selected all the street signs but at least I don't have to give up in frustration after 30 consecutive failed attempts