4 ms·
Neat project. How do you deal with the Tor latency? Doesn't it take ages to crawl that many pages? Looks like there's a bug with URLs, at least with onion.to.
by inahurry12345 13y ago
Neat project. How do you deal with the Tor latency? Doesn't it take ages to crawl that many pages?
Looks like there's a bug with URLs, at least with onion.to. Searching for 'google': https://kbhpodhnfxl3clb4.onion.to/?utf8=%E2%9C%93&q=google https://kbhpodhnfxl3clb4.onion.to/?utf8=%E2%9C%93&q=google gives me a link https://kbhpodhnfxl3clb4.onion.to/https://edramalpl7oq5npk.onion.to.to/Google https://kbhpodhnfxl3clb4.onion.to/https://edramalpl7oq5npk.o... which is clearly incorrect.
- IceyEC 13y agoYeah, it's taken a while to crawl ~100k pages; regarding that google search, it shouldn't have a problem, I just did the search again both in onion.to and in the actual app. May be a weird bug in onion.to, I saw a few things come through like that in the logs. Thanks for the feedback!