2 ms·
I recall the engine once actually picked up a genuine .onion domain (it was the Hidden Wiki, IIRC). It's safe to assume they do crawl the Tor space.
by vezzy-fnord 13y ago
I recall the engine once actually picked up a genuine .onion domain (it was the Hidden Wiki, IIRC).
It's safe to assume they do crawl the Tor space.
- gizmo686 13y agoIf they just pick up the .onion domain, they will try to crawl it and determine that the URL is incorrect, as .onion domains do not exist in the standard DNS stack. The only we for them to crawl Tor is if they went out of their way to crawl it (in which case they may or may use domains found on the standard web), or if they crawl domains like .onion.to, which behave like normal sites.