11 ms·

Tor has acknowledged their "botnet problem" since at least 2013: https://research.torproject.org/techreports/botnet-tr-2013-11-20.pdf https://research.torproje

Tor has acknowledged their "botnet problem" since at least 2013: https://research.torproject.org/techreports/botnet-tr-2013-11-20.pdf https://research.torproject.org/techreports/botnet-tr-2013-1... That same paper walks through the challenges of dealing with it and doesn't find any satisfactory solutions. As I wrote in our post on the topic, there's a trade off between security, anonymity, and convenience. CloudFlare provides security to our customers. We believe in the importance of anonymously accessing the Internet. Unfortunately, that means we have to sacrifice some convenience. If you haven't read it, I encourage you to see the post I wrote on the topic: https://blog.cloudflare.com/the-trouble-with-tor/ https://blog.cloudflare.com/the-trouble-with-tor/ The two long-term solutions we proposed — blinded tokens or CloudFlare supporting .onion addresses — we believe could reduce the inconvenience, but they'll require help from the Tor developers. While public posts like this are discouraging in terms of coming up with a better solution, I'm encouraged by private conversations we've had with Tor developers who acknowledge this is a hard problem and want to find solutions.

ryanlol 11y ago
>Tor has acknowledged their "botnet problem" since at least 2013: >https://research.torproject.org/techreports/botnet-tr-2013-1.. https://research.torproject.org/techreports/botnet-tr-2013-1.... >That same paper walks through the challenges of dealing with it and doesn't find any satisfactory solutions. But that's all utter nonsense! These botnets only access stuff inside the Tor network (i.e the C&C, that the operators want to hide), they don't use Tor to access clearnet content. Even a small botnet will have more nodes than there exists tor exit nodes (966 as of right now), what possible benefit could there be for the botnet operator from doing that?
zzzcpan 11y ago
Why don't you just drop IP-based reputation system for Tor IPs completely and develop something else for these IPs, something based on data from actual requests and responses? Because it sounds like you want to preserve an incorrect system and are pushing this problem on Tor.
- eastdakota 11y ago
  Quoting from our post (https://blog.cloudflare.com/the-trouble-with-tor/ https://blog.cloudflare.com/the-trouble-with-tor/): At CloudFlare we've not explicitly treated traffic from Tor any differently, however users of the Tor browser have been more likely to have their browsing experience interrupted by CAPTCHAs or other restrictions. This is because, like all IP addresses that connect to our network, we check the requests that they make and assign a threat score to the IP. Unfortunately, since such a high percentage of requests that are coming from the Tor network are malicious, the IPs of the Tor exit nodes often have a very high threat score. With most browsers, we can use the reputation of the browser from other requests it’s made across our network to override the bad reputation of the IP address connecting to our network. For instance, if you visit a coffee shop that is only used by hackers, the IP of the coffee shop's WiFi may have a bad reputation. But, if we've seen your browser behave elsewhere on the Internet acting like a regular web surfer and not a hacker, then we can use your browser’s good reputation to override the bad reputation of the hacker coffee shop's IP. The design of the Tor browser intentionally makes building a reputation for an individual browser very difficult. And that's a good thing. The promise of Tor is anonymity. Tracking a browser's behavior across requests would sacrifice that anonymity. So, while we could probably do things using super cookies or other techniques to try to get around Tor's anonymity protections, we think that would be creepy and choose not to because we believe that anonymity online is important. Unfortunately, that then means all we can rely on when a request connects to our network is the reputation of the IP and the contents of the request itself.
  - marshray 11y ago
    > But, if we've seen your browser behave elsewhere on the Internet acting like a regular web surfer and not a hacker, then we can use your browser’s good reputation to override the bad reputation of the hacker coffee shop's IP. Look, please correct me if I'm misunderstanding or taking your words out of context. But what I hear you saying is that CloudFlare is fundamentally opposed to user privacy at a business and an architectural level. I.e., if you don't agree to let CloudFlare track you around the web (perhaps by simply declining cookies) CloudFlare is likely to degrade your user experience to the point of being borderline unusable and then point the blame at you for coming from a bad network neighborhood.
jcoffland 11y ago
As a developer I will direct my clients away from CloudFlare services as long as CloudFlare continues this sort of attack on Tor which is ultimately an attack on privacy.
- themartorana 11y ago
  * facepalm * No room for nuance, huh? Or appreciation for the position CloudFlare is in and their obligation to their clients? How would you solve this? Abuse from Tor IPs is a known and documented problem. If you have a solution, I'll bet CloudFlare has a job opening.
  - tsukassa 11y ago
    If CloudFlare would at least let their client decide by themselves, that would be an awesome start. Even if it's enabled by default.
    - ultramancool 11y ago
      They do this already. See their blog post. You can explicitly whitelist Tor IPs, I thought it was plan limited to only enterprise, but it seems it's available to all actually.
      - lqdc13 11y ago
        It's a fundamental problem with IP-based reputation. Same goes for VPNs where there isn't a specific list of hosts you can whitelist.
        throwaway7767 11y ago
        IP based reputation has served us well, but it's no longer relevant. IPv4 has reached exhaustion, carrier-grade NAT is being deployed and the idea that an IP address correlates with one person or a very small group of people no longer holds. You can sort of pretend it does if you only serve america and europe, but that will change too. Any company that currently bases their offering on IP-based reputation better be working on different solutions to the problem or they're not going to stay relevant for long. This is an existential issue for CDNs.
tomjen3 11y ago
I don't use tor but I sometimes use a VPN which occasionally have the same problem. I don't tend to solve the capchas, I tend to use the back button - and I don't think I am alone. You should take this into account as well, since your customers are presumably not interested in losing business.
ultramancool 11y ago
Hi, I'm completely sympathetic to your problem - Tor is used by lots of spammers - totally understandable to try to prevent this spam from hitting your customers. But most of the services I run can't really be affected by this sort of spam (no public comment systems for example). I use CloudFlare on a few of my domains, if I don't care about bot traffic and just want to turn this CAPTCHA system off entirely, is there a way with just a pro account to do so? I appreciate the anti-DDoS protection and certainly having it automatically kick these CAPTCHAs on with only extremely high volumes of traffic could be appropriate, but enforcing them is just too much. I have security level set to minimum, but recently spun up a hidden service because users were still getting CAPTCHA'd over Tor.
- kristofferR 11y ago
  Yeah, I don't get why CloudFlare are so overaggressive with the captchas. The vast majority of captcha'd pages by CloudFlare on Tor which makes secure web browsing so cumbersome are completely read-only, while some may have a comment system hosted by a third party like Disqus and Facebook (and are therefore protected already). Other sites should have the captchas on a different level than the front page, like the login forms and other pages where write/spam access is somehow enabled. I can think of very few sites that are in need of a total ban on anonymous users/bots or that are write-access enabled by default without any login process.
  - grey-area 11y ago
    They also protect against ddos, sqli, overuse, bots etc. so comment spam is only one problem and get requests are not always safe. You'd think they could be far more sophisticated about reputation though and adjust it in realtime so that ips are by default trusted and are marked down temporarily for bad behaviour.
    - ascorbic 11y ago
      The problem is that with Tor the IP isn't much use as an identifier to establish reputation. As CloudFlare say in their post, they do use reputation of the exit node IPs to some extent, but the trouble is that there's so much abuse from Tor, most exit nodes always have bad rep. The zero knowledge tokens t hat they talk about are a way of establishing reputation without losing anonymity.
phw 11y ago
The bots in the "botnet problem" were connecting to a hidden service that was hosted in the Tor network. That isn't quite the same thing as bots that connect to hosts outside of Tor. Therefore, this isn't an issue for CloudFlare, and a bad example.
deleted 11y ago
[deleted]
throwaway7767 11y ago
The post from the Tor project does not state anywhere that Tor is not used by botnets or that it's used for malicious purposes. They specifically question your specific assertion that 94% of Tor traffic is malicious. I'm not surprised, it's quite a statement and it calls for some supporting evidence. Surely you can see how, given the amount of outreach they do to educate regular people about the positive uses of Tor, putting forth unfounded statements like that might be perceived negatively. I am glad that CloudFlare has put effort into this problem and as a Tor user I appreciate it (though obviously, this problem goes far beyond Tor, as mentioned in the article - your systems will have the exact same problems with large-scale IPv4 NAT, so really it's not optional for a CDN provider). But please, stick to facts when presenting the case.
- mirimir 11y ago
  Yes, that botnet dig is bullshit. Some botnets use Tor for C&C. But jerks don't need botnets to have lots of Tor exits. That's what VMs are for. Botnets give you lots of residential IPs.
  - baby 11y ago
    C&C? VMs? What?
    - mirimir 11y ago
      C&C = command and control [0] VM = virtual machine (local or remote VPS) [1] Let's say that I have a box with a couple quad-core Xeons and 64GB RAM, and a 100Mbps uplink. I can easily run 150-200 Debian VMs, each running one or more tor processes. [0] https://security.radware.com/ddos-knowledge-center/ddospedia/command-and-control-server/ https://security.radware.com/ddos-knowledge-center/ddospedia... [1] VPS = virtual private server
      - baby 11y ago
        Ah right, didn't think about running several Tor instances in VMs. I'm sure there are better ways to run Tor from VMs though, if you know how the protocol works just run instances of some program that runs it instead of running the whole thing in a VM.
yoo1I 11y ago
You still purposefully resolve spam domains on at least tim.ns.cloudflare.com and leah.ns.cloudflare.com and refuse do to anything about it. I see a certain hypocrisy in claiming to protect your customers, and at the same time enabling criminal operations through allowing them use of your infrastructure. (For the record and because you tell me this at every point of contact, I know your main business is reverse-proxy. I don't care. You run DNS infrastructure, you are responsible for it.)
- throwaway321412 11y ago
  Yep and on top of that they have for years they have allowed DDoS 'booters' to stay online, with the claim of "we have no way to remove the content, we're just a reverse proxy." If they're not actually hosting it, they think it's OK for whatever it is to pass through their network. http://www.crimeflare.com/damon.html http://www.crimeflare.com/damon.html
  - yoo1I 11y ago
    Aye, thanks for the write-up! I don't really get them either, surely they have enough paying customers to be able to afford some basic data hygiene. I'd really like to find some ways to make it more costly for them to keep the scum on their networks than to send out stupid "we r a reverse proxy!!!" replies, that clearly have nothing to do with anything. But yeah, keep complaining about Tor, good job CloudFlare /s
    - pfg 11y ago
      Let me get this straight. People are criticizing CloudFlare for inconveniencing Tor users - a tool which, among other things, can be used to fight censorship. At the same time, people are calling them out on their abuse policy which essentially boils down to "We won't take down sites based on content unless we receive a court order telling us to." That's interesting, to say the least.
      - yoo1I 11y ago
mirimir 11y ago
More than a hard problem, this is arguably an intractable problem. Tor is already vulnerable, and trading anonymity for convenience (or even appearing to do so) would be unworkable. There are other anonymity systems waiting in the wings, and they would arguably gain support if the Tor Project sold out to CloudFlare. The only viable approach for Tor, as I see it, is to treat CloudFlare as damage, to be routed around.

ryanlol 11y ago

>Tor has acknowledged their "botnet problem" since at least 2013: >https://research.torproject.org/techreports/botnet-tr-2013-1.. https://research.torproject.org/techreports/botnet-tr-2013-1.... >That same paper walks through the challenges of dealing with it and doesn't find any satisfactory solutions. But that's all utter nonsense! These botnets only access stuff inside the Tor network (i.e the C&C, that the operators want to hide), they don't use Tor to access clearnet content. Even a small botnet will have more nodes than there exists tor exit nodes (966 as of right now), what possible benefit could there be for the botnet operator from doing that?

zzzcpan 11y ago

Why don't you just drop IP-based reputation system for Tor IPs completely and develop something else for these IPs, something based on data from actual requests and responses? Because it sounds like you want to preserve an incorrect system and are pushing this problem on Tor.

eastdakota 11y ago
Quoting from our post (https://blog.cloudflare.com/the-trouble-with-tor/ https://blog.cloudflare.com/the-trouble-with-tor/): At CloudFlare we've not explicitly treated traffic from Tor any differently, however users of the Tor browser have been more likely to have their browsing experience interrupted by CAPTCHAs or other restrictions. This is because, like all IP addresses that connect to our network, we check the requests that they make and assign a threat score to the IP. Unfortunately, since such a high percentage of requests that are coming from the Tor network are malicious, the IPs of the Tor exit nodes often have a very high threat score. With most browsers, we can use the reputation of the browser from other requests it’s made across our network to override the bad reputation of the IP address connecting to our network. For instance, if you visit a coffee shop that is only used by hackers, the IP of the coffee shop's WiFi may have a bad reputation. But, if we've seen your browser behave elsewhere on the Internet acting like a regular web surfer and not a hacker, then we can use your browser’s good reputation to override the bad reputation of the hacker coffee shop's IP. The design of the Tor browser intentionally makes building a reputation for an individual browser very difficult. And that's a good thing. The promise of Tor is anonymity. Tracking a browser's behavior across requests would sacrifice that anonymity. So, while we could probably do things using super cookies or other techniques to try to get around Tor's anonymity protections, we think that would be creepy and choose not to because we believe that anonymity online is important. Unfortunately, that then means all we can rely on when a request connects to our network is the reputation of the IP and the contents of the request itself.
- marshray 11y ago
  > But, if we've seen your browser behave elsewhere on the Internet acting like a regular web surfer and not a hacker, then we can use your browser’s good reputation to override the bad reputation of the hacker coffee shop's IP. Look, please correct me if I'm misunderstanding or taking your words out of context. But what I hear you saying is that CloudFlare is fundamentally opposed to user privacy at a business and an architectural level. I.e., if you don't agree to let CloudFlare track you around the web (perhaps by simply declining cookies) CloudFlare is likely to degrade your user experience to the point of being borderline unusable and then point the blame at you for coming from a bad network neighborhood.

jcoffland 11y ago

As a developer I will direct my clients away from CloudFlare services as long as CloudFlare continues this sort of attack on Tor which is ultimately an attack on privacy.

themartorana 11y ago
* facepalm * No room for nuance, huh? Or appreciation for the position CloudFlare is in and their obligation to their clients? How would you solve this? Abuse from Tor IPs is a known and documented problem. If you have a solution, I'll bet CloudFlare has a job opening.
- tsukassa 11y ago
  If CloudFlare would at least let their client decide by themselves, that would be an awesome start. Even if it's enabled by default.
  - ultramancool 11y ago
    They do this already. See their blog post. You can explicitly whitelist Tor IPs, I thought it was plan limited to only enterprise, but it seems it's available to all actually.
    - lqdc13 11y ago
      It's a fundamental problem with IP-based reputation. Same goes for VPNs where there isn't a specific list of hosts you can whitelist.
      - throwaway7767 11y ago
        IP based reputation has served us well, but it's no longer relevant. IPv4 has reached exhaustion, carrier-grade NAT is being deployed and the idea that an IP address correlates with one person or a very small group of people no longer holds. You can sort of pretend it does if you only serve america and europe, but that will change too. Any company that currently bases their offering on IP-based reputation better be working on different solutions to the problem or they're not going to stay relevant for long. This is an existential issue for CDNs.

tomjen3 11y ago

I don't use tor but I sometimes use a VPN which occasionally have the same problem. I don't tend to solve the capchas, I tend to use the back button - and I don't think I am alone. You should take this into account as well, since your customers are presumably not interested in losing business.

ultramancool 11y ago

Hi, I'm completely sympathetic to your problem - Tor is used by lots of spammers - totally understandable to try to prevent this spam from hitting your customers. But most of the services I run can't really be affected by this sort of spam (no public comment systems for example). I use CloudFlare on a few of my domains, if I don't care about bot traffic and just want to turn this CAPTCHA system off entirely, is there a way with just a pro account to do so? I appreciate the anti-DDoS protection and certainly having it automatically kick these CAPTCHAs on with only extremely high volumes of traffic could be appropriate, but enforcing them is just too much. I have security level set to minimum, but recently spun up a hidden service because users were still getting CAPTCHA'd over Tor.

kristofferR 11y ago
Yeah, I don't get why CloudFlare are so overaggressive with the captchas. The vast majority of captcha'd pages by CloudFlare on Tor which makes secure web browsing so cumbersome are completely read-only, while some may have a comment system hosted by a third party like Disqus and Facebook (and are therefore protected already). Other sites should have the captchas on a different level than the front page, like the login forms and other pages where write/spam access is somehow enabled. I can think of very few sites that are in need of a total ban on anonymous users/bots or that are write-access enabled by default without any login process.
- grey-area 11y ago
  They also protect against ddos, sqli, overuse, bots etc. so comment spam is only one problem and get requests are not always safe. You'd think they could be far more sophisticated about reputation though and adjust it in realtime so that ips are by default trusted and are marked down temporarily for bad behaviour.
  - ascorbic 11y ago
    The problem is that with Tor the IP isn't much use as an identifier to establish reputation. As CloudFlare say in their post, they do use reputation of the exit node IPs to some extent, but the trouble is that there's so much abuse from Tor, most exit nodes always have bad rep. The zero knowledge tokens t hat they talk about are a way of establishing reputation without losing anonymity.

phw 11y ago

The bots in the "botnet problem" were connecting to a hidden service that was hosted in the Tor network. That isn't quite the same thing as bots that connect to hosts outside of Tor. Therefore, this isn't an issue for CloudFlare, and a bad example.

deleted 11y ago

[deleted]

throwaway7767 11y ago

The post from the Tor project does not state anywhere that Tor is not used by botnets or that it's used for malicious purposes. They specifically question your specific assertion that 94% of Tor traffic is malicious. I'm not surprised, it's quite a statement and it calls for some supporting evidence. Surely you can see how, given the amount of outreach they do to educate regular people about the positive uses of Tor, putting forth unfounded statements like that might be perceived negatively. I am glad that CloudFlare has put effort into this problem and as a Tor user I appreciate it (though obviously, this problem goes far beyond Tor, as mentioned in the article - your systems will have the exact same problems with large-scale IPv4 NAT, so really it's not optional for a CDN provider). But please, stick to facts when presenting the case.

mirimir 11y ago
Yes, that botnet dig is bullshit. Some botnets use Tor for C&C. But jerks don't need botnets to have lots of Tor exits. That's what VMs are for. Botnets give you lots of residential IPs.
- baby 11y ago
  C&C? VMs? What?
  - mirimir 11y ago
    C&C = command and control [0] VM = virtual machine (local or remote VPS) [1] Let's say that I have a box with a couple quad-core Xeons and 64GB RAM, and a 100Mbps uplink. I can easily run 150-200 Debian VMs, each running one or more tor processes. [0] https://security.radware.com/ddos-knowledge-center/ddospedia/command-and-control-server/ https://security.radware.com/ddos-knowledge-center/ddospedia... [1] VPS = virtual private server
    - baby 11y ago
      Ah right, didn't think about running several Tor instances in VMs. I'm sure there are better ways to run Tor from VMs though, if you know how the protocol works just run instances of some program that runs it instead of running the whole thing in a VM.

yoo1I 11y ago

You still purposefully resolve spam domains on at least tim.ns.cloudflare.com and leah.ns.cloudflare.com and refuse do to anything about it. I see a certain hypocrisy in claiming to protect your customers, and at the same time enabling criminal operations through allowing them use of your infrastructure. (For the record and because you tell me this at every point of contact, I know your main business is reverse-proxy. I don't care. You run DNS infrastructure, you are responsible for it.)

throwaway321412 11y ago
Yep and on top of that they have for years they have allowed DDoS 'booters' to stay online, with the claim of "we have no way to remove the content, we're just a reverse proxy." If they're not actually hosting it, they think it's OK for whatever it is to pass through their network. http://www.crimeflare.com/damon.html http://www.crimeflare.com/damon.html
- yoo1I 11y ago
  Aye, thanks for the write-up! I don't really get them either, surely they have enough paying customers to be able to afford some basic data hygiene. I'd really like to find some ways to make it more costly for them to keep the scum on their networks than to send out stupid "we r a reverse proxy!!!" replies, that clearly have nothing to do with anything. But yeah, keep complaining about Tor, good job CloudFlare /s
  - pfg 11y ago
    Let me get this straight. People are criticizing CloudFlare for inconveniencing Tor users - a tool which, among other things, can be used to fight censorship. At the same time, people are calling them out on their abuse policy which essentially boils down to "We won't take down sites based on content unless we receive a court order telling us to." That's interesting, to say the least.
    - yoo1I 11y ago

mirimir 11y ago

More than a hard problem, this is arguably an intractable problem. Tor is already vulnerable, and trading anonymity for convenience (or even appearing to do so) would be unworkable. There are other anonymity systems waiting in the wings, and they would arguably gain support if the Tor Project sold out to CloudFlare. The only viable approach for Tor, as I see it, is to treat CloudFlare as damage, to be routed around.