3 ms·
>> "Currently when I look, Google tells me that Mahalo has 356,000 pages indexed" I see 'Results 1 - 10 of about 2,200,000 from mahalo.com' Have things steppe
by axod 17y ago
>> "Currently when I look, Google tells me that Mahalo has 356,000 pages indexed"
I see 'Results 1 - 10 of about 2,200,000 from mahalo.com'
Have things stepped up a gear or am I misunderstanding?
- mvandemar 17y agoNo, different datacenters will show different results... sometimes very different. It also matters if you are visiting Google.com or one of the country variants. According to what Jason said in another comment, however, all of their pages are listed in their xml sitemap, and all of those are listed in a master xml sitemap index located here (warning! huge files if you follow the links in the first one!): http://www.mahalo.com/sitemapindex.xml http://www.mahalo.com/sitemapindex.xml Based on what I saw 2 million+ looks like a huge overestimate, if what Jason said is true. Edit: My bad, frederickcook's answer was the right one. I didn't realize you were doing a regular text search.
- chronomex 17y agoThere are 12 sitemap-mahalo.xml files, each with 50,000 urls in it. The 12th has 48,661 urls by my count. That gives a total size of 598,661 pages. Methodology: wget http://www.mahalo.com/sitemapindex.xml cat sitemapindex.xml |sed -e's/[<>]/\n/g'|grep ^http|xargs wget for P in sitemap-mahalo.xml* ; do echo $P ; cat $P |sed -e's/[<>]/\n/g'|grep ^http|wc -l ; done Edit: Formatting
- frederickcook 17y agoSearch "site:mahalo.com" Simply "mahalo.com" lists every indexed page with that text on it, such as this one.
- axod 17y agoYup that's what I did. Maybe the particular google datacenter I hit had a high estimate there. It also says 59,300,000 for scribd.com which I can't help but draw comparisons with this.