4 ms·

I'm out of the loop on this. What has been happening? Do you get IP blocked? Rate limited?

• realusername 3y ago
  You do and since the rise of Cloudflare, you need their approval to create your search engine
  • oceanplexian 3y ago
    Unfortunately for them, AI is a double edged sword. AI can trivially scrape content from a screencap. And there are thousands of residential proxies that can’t be defeated even by companies like CF.
    • spencerflem 3y ago
      If you have residential proxies, you don't need AI to scrape via video, you have the HTML already.
      • oceanplexian 3y ago
        Companies spend thousands of engineering hours building elaborate mechanisms to obfuscate JS, protect APIs, apps, trigger captchas, and hide content, all of it completely powerless against the current gen of AI tools. If a human can read it, a bot can.
        • Mr_P 3y ago
          Those same companies often invest in accessibility for vision-impaired users. I'm not sure you need a screen capture to scrape content when the site is designed to be navigable with a screen reader.
          • timthelion 3y ago
            Surprisingly chatgpt is not accessible via screen reader, their captcha is purely image based.
    • CyberDildonics 3y ago
      Is OCR from 30 years ago now "AI"?