3 ms·
It's the top link in the XML Sitemap that is linked from robots.txt. This is how Search Engines generally find new content on large sites, rather than by crawli
by sltkr 3y ago
It's the top link in the XML Sitemap that is linked from robots.txt. This is how Search Engines generally find new content on large sites, rather than by crawling.
i.e. https://www.phoronix.com/robots.txt https://www.phoronix.com/robots.txt contains:
Sitemap: https://www.phoronix.com/sitemap.php
And https://www.phoronix.com/sitemap.php https://www.phoronix.com/sitemap.php contains:
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://www.phoronix.com/review/raspberry-pi-5-benchmarks</loc>
<lastmod>2023-09-28</lastmod>
</url>
...
</urlset>
And generally search engines boost new content, which probably explains why it was among the top search results.