4 ms·

Yandex is a great example of a company which absolutely squashes Google on technical chops wherever it has tried to compete; and it only has 9000 employees (com

• rewq4321 6y ago
  Not entirely related to this discussion, but one thing I know about Yandex: Their web crawlers are ruthless. They hammer my sites to the point where I need to specifically throttle them based on their "from" header (they seem to ignore 429s - they simply make requests with a different IP). This is only a problem because I have a few sites that are like "search engines" in that they have basically an infinite number of pages (one for each possible search term), and so they just keep crawling the "related searches" links. It's probably not a problem for most sites, but it's a pain for me. Google seems to more strategically crawl pages based on search terms that they know are popular, whereas Yandex just crawls every possible link it can find. All of that said, I'm quite happy to deal with it for the benefit of having another competitor to Google - the more the merrier. Hopefully one day they add some more "smarts" to their crawlers though - starting with respecting 429s across all their IPs, perhaps.
  • scottlocklin 6y ago
    In modern Russia search engine search you. They actually answer their email; if you ask them they'll probably stop.