"What makes Pipl different?" leads me to: Unlike a typical search-engine, Pipl is designed to retrieve information from the deep web, our robots are set to inte

"What makes Pipl different?" leads me to: Unlike a typical search-engine, Pipl is designed to retrieve information from the deep web, our robots are set to interact with searchable databases and extract facts, contact details and other relevant information from personal profiles, member directories, scientific publications, court records and numerous other deep-web sources. But much of the "deep web" isn't indexed because indexing either involves ignoring robots.txt or explicitly going against a site's Terms of Use. For example, take Amazon, which Pipl holds up as an example of a site that has "deep web" content. Amazon's Terms of Use, available here http://www.amazon.com/gp/help/customer/display.html?nodeId=508088 http://www.amazon.com/gp/help/customer/display.html?nodeId=5... say: license does not include ... any use of data mining, robots, or similar data gathering and extraction tools. How does Pipl index the deep web then? Have they gotten permission from all these companies? The TechCrunch article notes "[t]he folks at Pipl were hesitant to discuss their “secret sauce” with me" and only provides vague details, but even those details seem to point towards some sort of automated crawling of parts of sites that those sites' Terms of Use usually forbid automated crawling of.