agree to disagree , but: Google already has access to any webpage because its own search Crawlers are allowed by most websites, and google crawls recursively.

agree to disagree , but: Google already has access to any webpage because its own search Crawlers are allowed by most websites, and google crawls recursively. Thus Gemini has an advantage of this synergy with google search. Perplexity does not crawl recursively (i presume -- therefore it does not need to consult robots.txt), and it doesn't have synergies with a major search engine.