4 ms·

I understand the user experience issue, but I'm trying to illustrate that there's also the issue of Google's resources. They are a business after all, no matter

by kadavy 16y ago

I understand the user experience issue, but I'm trying to illustrate that there's also the issue of Google's resources. They are a business after all, no matter how they hope to not be evil. What I did to make the speed improvements - check out the post that I referenced (twice) in the post: http://www.kadavy.net/blog/posts/wordpress-optimization-dreamhost-rackspace/ http://www.kadavy.net/blog/posts/wordpress-optimization-drea...

ericd 16y ago
I don't see how it factors into their resource usage at all. A crawler that's blocked waiting for a reply from your site is unlikely to be using much/any CPU time compared to an active crawler, and that time is surely being used by another crawler on that machine. All crawlers blocked and waiting? That means they can spin up 50 more! Unless they're doing it wrong, and I very much doubt they are given it's central to their business, it's purely a user experience issue.
- kadavy 16y ago
  I don't know the details of how their crawler works, but it seemed to me that if a page takes more time to serve to the crawler, there would be some lost resources. Even so, faster page load time also means higher AdSense CTR, and more money - though I guess you could make an indirect user experience case for that.
  - ericd 16y ago
    There's bound to be a small amount of memory used by the idle crawler, but if they designed it well, it seems unlikely that that would be a limiting factor. You make a fair point about the AdSense. I would guess they care more about a better user experience, but it's certainly possible. Everything they do can be viewed as a means to keep people searching and using the web, and clicking on more ads.
  - barmstrong 16y ago
    I just left a similar comment on the post - it likely wouldn't affect resources. The ranking bonus from fast loading pages is probably because Google knows humans prefer fast loading sites.

by kadavy 16y ago

ericd 16y ago

I don't see how it factors into their resource usage at all. A crawler that's blocked waiting for a reply from your site is unlikely to be using much/any CPU time compared to an active crawler, and that time is surely being used by another crawler on that machine. All crawlers blocked and waiting? That means they can spin up 50 more! Unless they're doing it wrong, and I very much doubt they are given it's central to their business, it's purely a user experience issue.

kadavy 16y ago
I don't know the details of how their crawler works, but it seemed to me that if a page takes more time to serve to the crawler, there would be some lost resources. Even so, faster page load time also means higher AdSense CTR, and more money - though I guess you could make an indirect user experience case for that.
- ericd 16y ago
  There's bound to be a small amount of memory used by the idle crawler, but if they designed it well, it seems unlikely that that would be a limiting factor. You make a fair point about the AdSense. I would guess they care more about a better user experience, but it's certainly possible. Everything they do can be viewed as a means to keep people searching and using the web, and clicking on more ads.
- barmstrong 16y ago
  I just left a similar comment on the post - it likely wouldn't affect resources. The ranking bonus from fast loading pages is probably because Google knows humans prefer fast loading sites.