I see crawlers as enablers - I've even gone so far as spinning up a dedicated server instance to serve content to the bots - so they get good response times whi

I see crawlers as enablers - I've even gone so far as spinning up a dedicated server instance to serve content to the bots - so they get good response times which increases relevancy in search results.