4 ms·

Honestly the user agent string should just be removed. The correct way to use it seems to be do not do anything with it so why does it still exist.

• spurgu 8y ago
  If you don't have an "officially recognized" UA string some websites will assume you're a scraper/crawler/bot/ and rightfully so.
  • threatofrain 8y ago
    That's just a heuristic of the times. Soon bots will be much smarter than that.
    • seba_dos1 8y ago
      They already are.
  • ndnxhs 8y ago
    If I was ever trying to scrape a website that blocked my user agent I would just set the user agent to chrome on windows. Every library I have used supports setting whatever UA you want.