2 ms·

After they ingested everything

• fragmede 3y ago
  We don't actually know what they trained on. If we want to trust them, they've always honored robots.txt, because why wouldn't they? On the other hand, if we want to believe they're as capricious as we want to make them out to be, they've got a copy of LibGen, and every single torrent out there to train it in on, on top of using robots.txt to find what you don't want crawled. robots.txt is great - it tells you where your evil scraper should go to get the good stuff.
  • __loam 3y ago
    Gpt-4 was released to the public on March 14th last year. They told people how to block their agent in August.
    • fragmede 3y ago
      yes but User agent: *