4 ms·

Yes. Did that after this episode.

• tokenizerrr 9y ago
  Were you seriously expecting bots to read your T&C? Or anyone, for that matter? Did you mention that it was okay for Google to scrape your site?
  • throwaway6845 9y ago
    We're not talking generic "bots". We're talking a custom scraper written for this site and this site only. Yes, I am expecting the people who spend hours inspecting the source of my site, and then writing a custom scraper for it, to spend 30 seconds reading the T&Cs first.
    • tokenizerrr 9y ago
      Not sure why you'd expect that. If my webbrowser can download your source code, my software will as well. If you want people to read it put your content behind a sign up with a checkbox.
      • throwaway6845 9y ago
        It is _already_ behind a sign-up with a checkbox. They scraped their way past that too.
        • tokenizerrr 9y ago
          Ah, that changes things somewhat.
        • samstave 9y ago
          How? (Seriously, how does one do this?)
          • kossae 9y ago
            Simply log in first, then perform the scrape programmatically. Seen here: https://kazuar.github.io/scraping-tutorial/ https://kazuar.github.io/scraping-tutorial/
            • samstave 9y ago
              oh... I thought they were able to circumvent logging in and could scrape directly... hat makes much more sense now, thank you...