21 ms·

Tell HN: Stack Exchange (Stack Overflow) is now blocking Tor

• PaulHoule 5y ago
  It's a general anti-abuse measure. It was more than ten years ago but I was seriously spamming one of those reddit-like sites for software developers. I made about 1000 fake accounts with what looked like real names and photographs but if somebody looked close they might have noticed I made no effort to match the gender of the name to the gender of the photograph. The accounts generated a good amount of "cover traffic" voting for articles that weren't mine but I would give one of my articles somewhere between 20 to 80 votes that would usually recruit an even larger number of real votes. Once in a while somebody would say something like "That's a pretty good blog but not that good, I can't see how he's possibly getting so many votes for every single post" My take was that what I was doing was pretty safe because my fake users were actually pretty well behaved and there wasn't going to be any real perception that there was a problem. Even so I was using Tor to make sure that my requests were not coming from the same IP address and it wouldn't be so obvious what I was up to. I lost the database with the fake accounts in a hard drive crash and I wasn't about to push the issue because I had also attracted some attention from the F.B.I. which knew I (or at least my alter ego) was making trouble with Tor so they tried to entrap me into distributing child porn over Tor. I told them no of course ("the FBI in my country sees that as an important enforcement priority") but that did make me lose interest in Tor. Back then Wikipedia would require you to log in to make any edits over Tor because they didn't want to deal with that kind of BS either. It's not unusual for many kinds of sites to have restrictions on Tor users for the same reason. My last covert web crawling project involved making a machine image that could be deployed in any AWS zone to do about 200 requests and then pop up in some other place. On one level it looks like you are getting hit from Japan and Singapore and Germany, and... But on another level it is all AWS. At some point they looked at their logs and freaked out and made changes to make it fundamentally harder to crawl out their site. We've got the running gag today that it's not safe for me to go to Delaware now. But practically a lot of sites block AWS addresses for the same reason they block Tor. It's just way too easy for a third rate hacker wannabee to seem to be coming from too many IP address that way.
• gerikson 5y ago
  There's probably been an issue with spam originating from that IP.
  • iqanq 5y ago
    I've tried to create new circuits, changing IP addresses, and they are all blocked.
  • yanmaani 5y ago
    That makes sense for posting, where Tor users often get a CAPTCHA etc, but I can't even read it. If I go to any site on SE, I'm hard blocked, I can't even fill in a captcha to bypass it.
    • paxys 5y ago
      Loading CAPTCHAs is still very expensive, and is not an answer to large-scale DDoS attacks. Those have to be handled higher up the stack, usually at the proxy/load balancer level.
  • dmitrygr 5y ago
    That’s a very convenient excuse, if your goal is to collect more data sell to advertisers and anonymous users annoy you. What next? There has been a lot of spam from people wearing clothes, so everybody needs to be strip searched?
    • hirako2000 5y ago
      that's pretty much how safety regulations at airports work. Stackoverflow perhaps hired a prior airport consultant for security.
  • stjohnswarts 5y ago
    I tried a new tor a few times, they all got blocked.
• fabian2k 5y ago
  There was no intentional blocking of Tor there in the past as far as I understand. But there are some mechanisms that block users from IP addresses that posted spam or abusive content. Those are automated and based on flags and not directed towards Tor specifically, but of course TOR addresses are more likely to have been misused that way so your chances of encountering these blocks is dramatically higher when using Tor. But these blocks only apply to low reputation users, I think once you're above 100 they don't matter anymore. And they also don't block reading the sites. There have been DDoS attacks recently on Stack Overflow and Stack Exchange, I think it's quite plausible those have something to do with this. They still seem to be under active attack intermittently.
  • Renaud 5y ago
    That doesn't explain why you can't read the site. OP is mentioning that they can't access the site at all.
    • fabian2k 5y ago
      That's why I suspect it's something new and might be related to the DDoS. The main spam and abuse problems are handles by the system I mentioned, which is not the reason here. So this is likely a defense against something else.
      • yanmaani 5y ago
        You're looking at simple, static pages. Unless something is horribly wrong with their infrastructure, these should take milliseconds to generate, if even that.
        • jasode 5y ago
          >You're looking at simple, static pages. Stackoverflow pages have many dynamic components like vote counts, reputation points, sidebar related-questions-links, new comments, etc. An excerpt from their blog explains they can't cache the output : https://nickcraver.com/blog/2019/08/06/stack-overflow-how-we-do-app-caching/#doing-more-harm-than-good https://nickcraver.com/blog/2019/08/06/stack-overflow-how-we... Even though Stackoverflow's website io access pattern has higher reads than writes, the resultant generated html is still not as static as Wikipedia pages. Even if they're efficiently using cpu to generate the dynamic elements, the cost of egress traffic may also be a factor. All that said, I don't have any insight into what heuristics they use to block certain ip addresses.
• uallo 5y ago
  My tiny side project gets a lot of bot traffic. And just yesterday, I had a large spike from TOR including the IP range 185.220.101.0/24. I was also considering blocking these IP addresses. As the project relies on third-party APIs which are rate-limited, the bots may cause larger loading times for real users. I'd assume that Stack Exchange might also get lots of bot traffic from TOR.
  • GentooInstaller 5y ago
    It was never "TOR", it's; Tor: https://matt.traudt.xyz/posts/2021-02-22-tor-spelling https://matt.traudt.xyz/posts/2021-02-22-tor-spelling. The DNS checking service is rate limited and adds latency, but there is a bulk exit list available here; https://check.torproject.org/torbulkexitlist https://check.torproject.org/torbulkexitlist (please don't use 3rd party blocklists, as those often contain middles and guards, even though traffic cannot leave those kind of relays). But, before you block Tor, please consider that doing so will most likely block a number of legitimate users who you haven't noticed before - just to stop one jerk. Instead, you could restrict access to certain parts of your site for Tor visitors for example. Bot traffic from Tor is usually of the easily detectable variety (as the attacker didn't have enough skill to build/acquire a botnet and get "clean" IP's),
    • bartvk 5y ago
      Thank you for correcting the spelling.
  • yanmaani 5y ago
    A middle-ground might be just putting some kind of CAPTCHA or similar on them, if they're causing actual trouble.
    • wanderer_ 5y ago
      Or a regular old rate limiter - who's loading a new page more often than a couple of seconds anyway?
  • goodpoint 5y ago
• floatingatoll 5y ago
  Tor provides anonymity without accountability; having some doors closed to all Tor users is the price paid for that anonymity. If that price isn’t acceptable, either modify Tor to allow more granular accountability in some privacy-protecting way, or don’t use Tor when accessing services that are closed to it.
  • yanmaani 5y ago
    I can use more or less all other websites with Tor fine, though. It's only Stack Overflow that insists on this nonsense. On most other sites, including this one, I can even make an account and post. Is their site really so sensitive as to make reading with Tor impossible?
    • rmbyrro 5y ago
      Good question. They might be trying to fight SPAM or else. Could just block posting, commenting and up/down voting...
    • james-redwood 5y ago
      Tor generally doesn't work on Google services.
      • jbentham 5y ago
        Is this actually true? I've often used Tor to access Google Docs and Google Maps and to my knowledge have never had a problem. In fact, I'm not even presented with Captchas.
        • bigyellow 5y ago
      • zozbot234 5y ago
        YMMV. Google Search might as well be blocked completely, I guess they don't want to deal with all the SEO-targeting search queries that would otherwise come from Tor.
• waiwai933 5y ago
  > I tried to contact them by e-mail and they didn't bother to respond, as is par for the course these days. Given you only contacted them today presumably, I'd allow them at least a few days to respond...
  • yanmaani 5y ago
    In my experience, when it comes to big players, responses are either bimodal - either they respond within a few hours, or not at all. That said, if they respond, I'll edit the post.
    • Jcowell 5y ago
      As someone who works close to the SO office. No. I’d expect everyone to be home asleep at that time.
    • MisterSandman 5y ago
      Stack Overflow is far from """huge""" Also, dude, it was the end of the work done or close to midnight in most of the western world when you made this post.
  • deleted 5y ago
    [deleted]
• chaoz_ 5y ago
  Maybe Stackoverflow finally started to aggressively block everything that might "crawl" their data. I'm super tired of all the paraphrased/translated copies in Google results.
  • 1337shadow 5y ago
    Which don't even link to original contents to look like their content is actually original... Do something about it google please!
  • petercooper 5y ago
    PSA: Anyone who needs a large amount of Stack Overflow data for whatever (legal) reason, they share dumps at https://archive.org/details/stackexchange https://archive.org/details/stackexchange – and perhaps elsewhere.
    • fao_ 5y ago
      The license is CC-BY-SA 4.0, so the sites that host copies of their data verbatim are (arguably) legal.
      • peschkaj 5y ago
        Yup! Mostly legal. We maintain a legal guidance page[1] covering what you can and can't do with the data. I assume our lawyers have been involved, but what do I know? [1]: https://stackoverflow.com/legal/trademark-guidance https://stackoverflow.com/legal/trademark-guidance
      • zozbot234 5y ago
        It depends. The "attribution" requirements can be a bit onerous at times, in ways that might put the most casual and opportunistic reusers of even CC-BY-SA data at some legal risk. (It's of course easier for a legitimate reuser with an actual stake in the data to dot the i's and cross the t's.)
  • southerntofu 5y ago
    The problem is with Google not with StackOverflow letting people read their stuff. A search engine which does not prioritize pages with advertisements and broken Google-powered translations (yes the french version of those SO/GH sites are hilarious) will filter that crap out. I mean anyone who'd like to crawl/scrap a website can do it from home, with the help of friends, or by renting "resential VPN" services for a few bucks a month. Blocking Tor does not prevent scrapping.
• 02020202 5y ago
• cletus 5y ago
  This isn't necessarily deliberate. Example: years ago at Google we had proxies for Web traffic. Around lunch time you could hit an issue with HN where the traffic was blocked for too many visits from the same IP. There were thousands of engineers and they were concentrated on 1-2 proxy IPs. A lot of systems tackle potential abuse (eg brute forcing passwords, DoS attacks) with a token and bucket algorithm. Example: a given source IP might get 20 tokens. Each visit uses a token and those tokens are regenerated 1 per 2 seconds. So with a Tor exit node you may just be hitting abuse protection and that could happen on the CDN, gateway servers, internal servers and so on.
  • miohtama 5y ago
    Also, as a user the solution is to use high quality VPN, not Tor or any free VPN that are ripe for abuse.
• kragen 5y ago
  Is there a good alternative site built on the Stack Exchange data dumps? Open-source projects like the Stack Exchange knowledge base and Wikipedia are always at risk of suffering a hostile takeover by their hosting services, whatever the license may say, as recently happened in the spectacular case of Freenode, and happened in a less severe fashion earlier with Sourceforge injecting malware into downloads. Things like Deletionpedia and Wikiwand provide an essential service by reducing the cost of switching and thus the power of hosting services like Wikimedia, but I don't know of anything comparable for Stack Exchange.
• thomassmith65 5y ago
  @dang? Can someone please fix the title? Something like "I have been unable to connect to Stack Overflow using Tor today" would be appropriate. The title at the moment is rash speculation. One doesn't expect HN to be perfect, but one expects more conscientiousness than this.
  • float4 5y ago
    I disagree. I just opened Tor myself to check, and indeed every circuit (I've refreshed ~10 times using cmd+shift+L) results in an error page that literally states: "Reason: Blocked."
    • thomassmith65 5y ago
      It's beside the point. What if you had connected? What if an endpoint with a different IP can connect? The standard here ought to be that the title accurately reflect the content. This post does not. I don't take HN as the kind of place where people want to read speculation masquerading as fact.
      • float4 5y ago
        The core of your argument is that you don't think many reloads, by different users, from different Tor IP's, is convincing evidence, and I think that makes you part of a very tiny minority.
        • thomassmith65 5y ago
          It probably does, sadly.
        • stjohnswarts 5y ago
          I had the same result, 8 connects from "new identity" feature, all ips blocked. It may be temporary though, but today, for sure they are blocking tor.
          • 5y ago
• zozbot234 5y ago
  This is regrettable. I'm not going to even browse, much less plan to answer questions on Stack Exchange sites if this requires broadcasting privacy-sensitive personal information to them such as an IP address. (I'm ofc. OK with answering from my household or job IP address as a basic matter of accountability and anti-abuse, but surely not with browsing from there!) It's fine for Stack Exchange to want to manage Tor Browser use of their network, but then they should at least publish an .onion alternative in the HTTP response headers, so that Tor Browser users have a tolerable alternative to hitting their clearnet sites.
  • throwaway2037 5y ago
    Would a VPN be sufficient for your needs? I assume you will say that VPN is too insecure. Zero shill here: I heard the VPN offered by Mozilla recently received a security audit.
• tiffanyh 5y ago
  Off topic: does anyone understand why SO is shutting down their Jobs Board? I get they were acquired, but why would you want to shutdown a revenue generating business has been able to uniquely capture the attention of a highly skilled worker who's difficult to hire.
• binarymax 5y ago
  • edge17 5y ago
    I am not familiar with Prosus but wikipedia says it's Dutch, not Chinese.
    • eynsham 5y ago
      Prosus is owned by Naspers (RSA) which owns quite a lot of Tencent, so there is a Chinese connexion, but I’m not sure about Chinese control. I haven’t read much of News24 (owned by Naspers) but they didn’t seem to be that different coverage-wise from the rest of the SA media on the spy scandal—after they broke the story—although, good lord, Xiaomei Havard must have been the world’s worst spy if the story was right, as Rebecca Davis pointed out: https://www.dailymaverick.co.za/article/2021-09-01-how-seriously-should-we-take-the-claim-of-a-possible-chinese-spy-in-parliament/ https://www.dailymaverick.co.za/article/2021-09-01-how-serio....
      • Godel_unicode 5y ago
        Yes, but that Chinese connection is in the other direction; Naspers owns them, not the other way around.
        • eynsham 5y ago
          Indeed. I suppose that there is a blackmail risk: their stake could be made worthless unless Naspers has Prosus have the Stack Exchange people ban Tor, but I rather doubt this happened.
          • missedthecue 5y ago
            How could Tencent selectively make the shares that Naspers owns worthless?
• fabian2k 5y ago
  This is the official response from SE: https://meta.stackexchange.com/questions/376060/update-on-the-ongoing-ddos-attacks-and-blocking-tor-exit-nodes https://meta.stackexchange.com/questions/376060/update-on-th... For the past month Stack Overflow has been hit by weekly DDoS attacks that progressively grew in size and scope. In each incident, the attacker(s) have been changing their methodology and responding to our countermeasures. Initially we were able to detect and mitigate the attacks before any performance degradation could be noticed but the latest attacks ramped up very quickly and the site was brought down before we could react. While we cannot go into specifics on each attack in order to maintain opsec and not tip off the attackers, we can say that each individual attack has been using different IP addresses and targeted different aspects of the site. During an outage, our top priority is always getting the site back up and running. After traffic has been stabilized, we perform a post mortem for the incident where we assess and improve upon the actions we have taken. During the outage last Sunday, we noticed that a large amount of the DDoS traffic originated from Tor exit nodes. The decision to block Tor exit nodes did not come lightly, in fact Teresa, our CTO was on the call when we discussed remediation methods. Due to the persistent nature of the attack and our desire to bring the site back up as fast as possible we made the decision to block all DDoS traffic endpoints, including these Tor exit nodes. We did not target, nor set out to block all traffic from Tor, that’s not something Stack has ever done. However, due to the shared nature of Tor exit nodes, some of them were also routing DDoS attacks to our sites and were blocked. We have tried removing these blocks between attacks but this action has resulted in further site outages as DDoS efforts continue to originate from these exit nodes. Unfortunately blocking the Tor exit nodes also blocks legitimate users from using them. An immediate solution for users who find themselves blocked is to access our site from other IP addresses, via home internet, work internet, or other VPN services. We are continuing to evaluate the situation and will keep our community updated. Thank you for your patience and understanding.
  • rafale 5y ago
    I wonder what's the motive to target StackOverflow? A disgruntled ex-employee?
    • colinmhayes 5y ago
• ravenstine 5y ago
  This is another reason why my interest shifted to I2P over Tor. Tor is designed more as a proxy to the cleaner, but I2P is focused on internal services, this it would take extra steps to use it for DOS attacks against plain ok websites. If you want to be available to I2P users, you basically opt-in to the network, as opposed to Tor's effectively opt-out approach. I2Ps design is supposedly better at dealing with DDOS attacks, but I don't exactly know the specifics of how that is. Overall, I think Tor will decline in usage over the next decade because it's too easy to use it nefariously and more institutions will find ways to block it.
  • Trockiy 5y ago
    There are onion sites in Tor. And no scalability issues of i2p (mentioned in their FAQ).
    • ravenstine 5y ago
      Onion sites are pretty secondary. Very few people using Tor know how to set them up or are even acutely aware of their existence. In I2P there is no concept of an exit node; that would only be unofficially possible by creating an eepsite that forwards connections from the clearnet. This is why I2P is overall a better model because it is a totally opt-in network but with no formal exit node distinction. I haven't read about I2Ps scaling issues in a while, but I'm pretty sure that the I2P network design is better at handling DDOS.
  • rany_ 5y ago
    > more institutions will find ways to block it You could detect and block Tor traffic by blocking the exit node IPs. They provide an API that lets you get all exit node IPs so that you could block them: https://check.torproject.org/torbulkexitlist https://check.torproject.org/torbulkexitlist They are putting no effort in making this hard for site admins.
• deleted 5y ago
  [deleted]
• throwaway5486nv 5y ago
  The solution is to not depend on stackexchange. Since the data of stackexchange is released under the creative commons license, third party can host it on onion site. The downside to this approach is real time access and posting questions will not be possible but at least one can browse the older posts.