3 ms·
There was a discussion some days ago about one of the AI companies using a very characteristic user agent string for web crawling, but a more browser-like one f
by mananaysiempre 2y ago
There was a discussion some days ago about one of the AI companies using a very characteristic user agent string for web crawling, but a more browser-like one for web browsing performed at the behest of the user. And there were some pertinent points there—if the AI bot is acting on an explicit request of a user, it does deserve to get treated like any other user agent more or less.
- PeterisP 2y agoIt also would make some sense for various real-time-request-augmented bots to not only use the user agent string of the user's browser, but actually use the user's browser to make the request.