HN replies sometimes (often) include statements unsupported by evidence For example, something like "90% of the current internet is behind Cloudflare, Google,

HN replies sometimes (often) include statements unsupported by evidence For example, something like "90% of the current internet is behind Cloudflare, Google, Akamai, etc." The www is only subset of the internet and "90% of the internet" does not use a CDN In any event, even if "internet" means "www" there is no evidence to support this statement https://www.netcraft.com/blog/may-2025-web-server-survey https://www.netcraft.com/blog/may-2025-web-server-survey Further, being "behind Cloudflare, Google, Akamai, etc." does not necessarily mean a User-Agent header is required For example, https://www.cloudflare.com https://www.cloudflare.com does not require a UA header https://example.com https://example.com is "behind CF" and does not require a UA header https://www.google.com https://www.google.com does not require a UA header https://developers.akamai.com https://developers.akamai.com does not require a UA header And so on Individual www users, unless doing mass scanning or the like, probably do not access 100% of the www's sites (according to Netcraft, most are inactive) More likely, individuals access only a fraction of the active sites It's unlikely that individuals access exactly the same list of websites^1 For me personally, the vast majority of sites that I access do not require a UA header The sample set I use as evidence to support comments I make about this on HN is a list of all sites submitted to HN over a recent period.^2 Currently this is 12,641 sites 1. To have a meaningful discussion about UA header requirements, a sample set of sites must be agreed upon 2. Perhaps HN readers access these sites and possibly access some of the same sites. Of course it's also possible that there are HN commenters who do not actually access HN submissions and instead only read comment threads and submit replies https://news.ycombinator.com https://news.ycombinator.com does not require a UA header