“Most real world sites” and “most real world sites you may want to scrape” are different, especially when weighted by information volume, so your selective quot

“Most real world sites” and “most real world sites you may want to scrape” are different, especially when weighted by information volume, so your selective quoting doesn’t help. Alexa top 100 probably contain more information, especially new information, than the rest of Alexa top 10000 combined (random guess I pulled out of my ass, don’t quote me on that), so the overwhelmingly scraping-resistant Alexa top 100 is what most scraping effort is directed against. Anyway, I’m not interested in a pedantic debate. Most (but not all) people who have attempted to scrape any popular site, or unpopular site behind Cloudflare at above-minimum protection level, in recent years should know exactly what I’m taking about. Edit: In case you’re not aware, many of the sites I have in mind are/were capable of defeating puppeteer-extra-plugin-stealth. User agent is only the most basic thing, like level 1 in a hundred-level dungeon.