W3.org used to have up to 130M (!) requests per day for HTML DTDs [1], probably mostly from poorly written spiders traversing all the links they managed to find

W3.org used to have up to 130M (!) requests per day for HTML DTDs [1], probably mostly from poorly written spiders traversing all the links they managed to find the crawled HTML pages, or clients validating schemas without any caching of DTDs. Would be indeed interesting to get reliable info about how example.com looks in comparison. [1] http://www.w3.org/blog/systeam/2008/02/08/w3c_s_excessive_dtd_traffic/ http://www.w3.org/blog/systeam/2008/02/08/w3c_s_excessive_dt...