4 ms·

About two years ago, spam and abuse were firmly among Justin.tv's biggest problems. It turned out the worst offenders were using tor for all network traffic. Un

About two years ago, spam and abuse were firmly among Justin.tv's biggest problems. It turned out the worst offenders were using tor for all network traffic. Unfortunately just configuring all of our servers to reject traffic from tor was by far the biggest bang-for-buck thing we did to kill the abuse. If there's a better way, I'd love to know about it.

mike-cardwell 16y ago
When Wikipedia had this problem they just blocked edit access from Tor exit nodes: https://secure.wikimedia.org/wikipedia/meta/wiki/Editing_with_Tor https://secure.wikimedia.org/wikipedia/meta/wiki/Editing_wit... Seems like a more measured response than outright blocking of everything.
avar 16y ago
Well, firstly this isn't a big problem for HTTP based services like justin.tv, almost all Tor exit nodes allow HTTP exits, but it's a bigger issue for non-HTTP exits. But the better way is to take into account exit policies. Only ban access from those IPs that are Tor exits and allow access through to justin.tv. You can query the exit policies of a given exit with TorDNSEL: https://www.torproject.org/tordnsel/ https://www.torproject.org/tordnsel/ I also whipped up a quick script that can be fed a list of Tor exit nodes on STDIN and will spew out a list of exits that aren't allowed to access a given IP:PORT: http://gist.github.com/523328 http://gist.github.com/523328 According to that 5/1338 Tor exits have policies that don't allow exits to justin.tv on port 80 (199.9.249.22:80), but e.g. 309/1338 ban exits to irc.freenode.org:6667. That means that justin.tv is needlessly banning ~0.4% of Tor exit nodes, but someone using the same approach with an IRC server is needlessly banning ~23%. With SMTP (checked on a random GMail server) that percentage rises to 98.8%.
- seabee 16y ago
  Excuse me if I'm being dense; obviously it is unnecessary, but what's the drawback of banning a Tor exit node that disallows access to your IP? It just seems far easier to ban them all.
  - avar 16y ago
    Yes it's easier. It's also easy to just ban all of China. But the drawback is that you're needlessly banning people who aren't doing you harm, and discouraging others from running Tor exit nodes with limited exit policies. E.g. I'd run a Tor exit node with policies that only allowed wikileaks.org and a few other selected sites, but I don't. Because I know it'll cause me more trouble than it's worth due to a bunch of overzealous administrators.
- chc 16y ago
  Who cares if you ban an IP that's already unable to access your site? There's no downside, and it sounds considerably simpler than querying exit nodes.
  - ElliotH 16y ago
    Because you affect the person running the exit node - not just the people exiting from their IP.
    - moxiemk1 16y ago
      How? If the people couldn't make it to your IP anyway...
      - rdl 16y ago
        The assumption is that some (crazy) people are running tor exit nodes on IPs shared with other traffic. That is IMO a seriously bad practice; even putting stuff like that on the same /24 as critical servers is probably a bad idea.
        avar 16y ago
        The topic of this thread is that if you're running a Tor exit with a limited exit policy you'll get banned by lazy admins who presume that all Tor exit nodes have a promiscuous exit policy. There's theoretically nothing crazy or bad about running a Tor exit node like that, e.g. one that only allows exits to a few sites that you yourself control. But in practice it's bad for you because of people that think "let's just nuke it from orbit" is responsible behavior on the Internet, even going so far as to ban whole /24 networks.
        _delirium 16y ago
        Part of the goal of Tor is to have a bunch of normal people running Tor nodes on their own personal machines, VPSs, etc., in which case they're almost always shared with other traffic (the person's own traffic).
  - 16y ago
- abstractbill 16y ago
  Interesting stuff, thanks. One correction though, justin.tv involves much more than HTTP. Our chat system is actually an irc network, and then of course we have the whole video system which is all non-HTTP traffic. Right now we just block all tor traffic, for all ports. Sounds like perhaps we could do a bit better than that, and I'll let people know, but I can't say I think it will be a high priority for us unfortunately.
lzw 16y ago
If you're blocking access to your site from specific IPs or ranges of IPs, then you fail at the internet. If you have "IT People" who advocate such a policy, fire them on the spot, and then give them two hours to think about what they have said and plead for their job back. I find it ironic that tor is routing around the internet damage from censorship, only to run into internet censorship from damaged people.

mike-cardwell 16y ago

When Wikipedia had this problem they just blocked edit access from Tor exit nodes: https://secure.wikimedia.org/wikipedia/meta/wiki/Editing_with_Tor https://secure.wikimedia.org/wikipedia/meta/wiki/Editing_wit... Seems like a more measured response than outright blocking of everything.

avar 16y ago

Well, firstly this isn't a big problem for HTTP based services like justin.tv, almost all Tor exit nodes allow HTTP exits, but it's a bigger issue for non-HTTP exits. But the better way is to take into account exit policies. Only ban access from those IPs that are Tor exits and allow access through to justin.tv. You can query the exit policies of a given exit with TorDNSEL: https://www.torproject.org/tordnsel/ https://www.torproject.org/tordnsel/ I also whipped up a quick script that can be fed a list of Tor exit nodes on STDIN and will spew out a list of exits that aren't allowed to access a given IP:PORT: http://gist.github.com/523328 http://gist.github.com/523328 According to that 5/1338 Tor exits have policies that don't allow exits to justin.tv on port 80 (199.9.249.22:80), but e.g. 309/1338 ban exits to irc.freenode.org:6667. That means that justin.tv is needlessly banning ~0.4% of Tor exit nodes, but someone using the same approach with an IRC server is needlessly banning ~23%. With SMTP (checked on a random GMail server) that percentage rises to 98.8%.

seabee 16y ago
Excuse me if I'm being dense; obviously it is unnecessary, but what's the drawback of banning a Tor exit node that disallows access to your IP? It just seems far easier to ban them all.
- avar 16y ago
  Yes it's easier. It's also easy to just ban all of China. But the drawback is that you're needlessly banning people who aren't doing you harm, and discouraging others from running Tor exit nodes with limited exit policies. E.g. I'd run a Tor exit node with policies that only allowed wikileaks.org and a few other selected sites, but I don't. Because I know it'll cause me more trouble than it's worth due to a bunch of overzealous administrators.
chc 16y ago
Who cares if you ban an IP that's already unable to access your site? There's no downside, and it sounds considerably simpler than querying exit nodes.
- ElliotH 16y ago
  Because you affect the person running the exit node - not just the people exiting from their IP.
  - moxiemk1 16y ago
    How? If the people couldn't make it to your IP anyway...
    - rdl 16y ago
      The assumption is that some (crazy) people are running tor exit nodes on IPs shared with other traffic. That is IMO a seriously bad practice; even putting stuff like that on the same /24 as critical servers is probably a bad idea.
      - avar 16y ago
        The topic of this thread is that if you're running a Tor exit with a limited exit policy you'll get banned by lazy admins who presume that all Tor exit nodes have a promiscuous exit policy. There's theoretically nothing crazy or bad about running a Tor exit node like that, e.g. one that only allows exits to a few sites that you yourself control. But in practice it's bad for you because of people that think "let's just nuke it from orbit" is responsible behavior on the Internet, even going so far as to ban whole /24 networks.
      - _delirium 16y ago
        Part of the goal of Tor is to have a bunch of normal people running Tor nodes on their own personal machines, VPSs, etc., in which case they're almost always shared with other traffic (the person's own traffic).
- 16y ago
abstractbill 16y ago
Interesting stuff, thanks. One correction though, justin.tv involves much more than HTTP. Our chat system is actually an irc network, and then of course we have the whole video system which is all non-HTTP traffic. Right now we just block all tor traffic, for all ports. Sounds like perhaps we could do a bit better than that, and I'll let people know, but I can't say I think it will be a high priority for us unfortunately.

lzw 16y ago

If you're blocking access to your site from specific IPs or ranges of IPs, then you fail at the internet. If you have "IT People" who advocate such a policy, fire them on the spot, and then give them two hours to think about what they have said and plead for their job back. I find it ironic that tor is routing around the internet damage from censorship, only to run into internet censorship from damaged people.