2 ms·
That makes no sense. It’s still automated and not an actual human, and it’s still distilling information from the internet.
by stingraycharles 3y ago
That makes no sense. It’s still automated and not an actual human, and it’s still distilling information from the internet.
- rolisz 3y agoYour browser is also automating things, you're not resolving DNS and writing all the http requests by hand.
- worthless-trash 3y agoI have found the next topic for my blog post. Challenge accepted!
- jtbayly 3y agoSo because something is automated somewhere, nobody has to follow robots.TXT anymore?
- Spivak 3y agoYou have it backwards, when a human initiates the request, like clicking on a link, your browser (rightfully) will ignore robots.txt. Unattended requests like scrapers do respect robots.txt. This is a case where ChatGPT is acting more like a browser with a funny way of displaying the final result. Each request is initiated by a human so it would likely be reasonable to ignore it in this case.