5 ms·
Hmm, the robots.txt, IP blocking, and user agent blocking are all policies chosen by the web server hosting the data. If web admins choose to block Google compe
by 8organicbits 2y ago
Hmm, the robots.txt, IP blocking, and user agent blocking are all policies chosen by the web server hosting the data. If web admins choose to block Google competitors, I'm not sure that's on Google. Can you clarify?
- deleted 2y ago[deleted]
- GreenWatermelon 2y agoA nice example is the recent reddit-google deal which gives google' crawler exclusive access to reddit's data. This just serves to give google a competitive advantage over other search engine.
- not2b 2y agoWell yes, the Reddit-Google deal might be found to violate antitrust. Probably will, because it is so blatantly anticompetitive. But if a publication decides to give special access to search engines so they can enforce their paywall but still be findable by search, I don't think the regulators would worry about that, provided that there's a way for competing search engines to get the same access.
- arrosenberg 2y agoWhich is it? Regulators shouldn’t worry, or we need regulations to ensure equal access to the market?
- immibis 2y agoregulators wouldn't worry if all search engines had equal access, even if you didn't because you're not a search engine
- arrosenberg 2y agoAnd if I had wheels, I would be a car. Theres no equal access without regulation.
- sroussey 2y agoNope. That deal was for AI not search.
- darkwater 2y agoAntitrust kicks in exactly in cases like this: using your moat in one market (search) to win another market (AI)
- dns_snek 2y agoThis is false, the deal cuts all other search engines off from accessing Reddit. Go to Bing and search for "news site:reddit.com" and filter results by date from the past week - 0 results. https://www.404media.co/google-is-the-only-search-engine-that-works-on-reddit-now-thanks-to-ai-deal/ https://www.404media.co/google-is-the-only-search-engine-tha...
- wahnfrieden 2y agoWhat do you think search is