Ahrefs crawls the internet just like Google does. SeoMoz does the same: http://moz.com/help/pro/what-is-rogerbot- http://moz.com/help/pro/what-is-rogerbot- Eve

Ahrefs crawls the internet just like Google does. SeoMoz does the same: http://moz.com/help/pro/what-is-rogerbot- http://moz.com/help/pro/what-is-rogerbot- Every service that builds a backlink database uses their own crawlers So to answer the question on how they get their data so fast, they OWN the data and they use big hadoop farms to crunch it and prepare it for api retrieval.