3 ms·
“We’re also hopeful that Google, the world’s largest search engine and most wasteful user of Internet resources, will adopt IndexNow or a similar standard and l
by taylorfinley 4y ago
“We’re also hopeful that Google, the world’s largest search engine and most wasteful user of Internet resources, will adopt IndexNow or a similar standard and lower the burden of search crawling on the planet.” Pretty blunt language from CloudFlare!
- deleted 4y ago[deleted]
- JPLeRouzic 4y agoFrom the point of view of a little web site owner, Google certainly has a weird behavior. It comes several times per day (sometimes 20 times per day) even if a robot.txt tells to come only every 7 days (it's a blog with news every four days on average). Petalsearch (Huawei) indexing rate is half of Google, Apple and Bing half of Petalsearch, Yandex half of Bing. The other search engines respect the robot.txt. I wonder how much energy and Internet capacity is wasted because of Google and PetalSearch indexing.