4 ms·

reddit blocked datacenter IPs even before this change.

• nomilk 2y ago
  Could a motivated scraper not buy IPs/proxies that aren’t in those ranges, i.e. to blend in with general users?
  • xeromal 2y ago
    Just like every security feature in the physical and digital worlds, security just inconveniences honest people and the cost to bypass reduces the amount of people who try. Eventually it becomes expensive to scrape reddit's data and most people will stop.
  • Manuel_D 2y ago
    Proxy IPs are also known and typically blocked. In fact, you can't even browse reddit without logging in when connected to most proxies. Many web scraping companies have loads of phones hooked up in a rack in order to use mobile IPs. Companies can't just block mobile IPs because their site would become unusable for several city blocks (mobile IPs often correspond to a specific cell tower). This is the face of modern web scraping: https://i.imgur.com/U2RXi5G.jpeg https://i.imgur.com/U2RXi5G.jpeg
    • BeFlatXIII 2y ago
      Or, if you're on the shadier parts of the web, hacked IoT devices put to good use instead of DDoS zombies.