4 ms·

We've seen many discussions around the decreasing quality of Google in the past week. This part summarizes the conflict of interest between advertisers and sear

We've seen many discussions around the decreasing quality of Google in the past week. This part summarizes the conflict of interest between advertisers and search engines very well: "Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is "The Effect of Cellular Phone Use Upon Driver Attention", a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving. This search result came up first because of its high importance as judged by the PageRank algorithm, an approximation of citation importance on the web [Page, 98]. It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers."

streamofdigits 5y ago
Can a single centralized engine / business model ever balance all those conflicts? It feels that the time for decentralized search where entities are both contributing and benefiting from shared search indexes and have the ability to tweak their preferences is, broadly speaking, the space we need to look into?
- bborud 5y ago
  Centralized vs distributed represents a non-trivial set of technological challenges and differences. Those challenges tended to be poorly understood by anyone who hadn't given the problem of designing a search engine much thought in the past. I don't know if that has improved since I was part of the search engine industry. It probably has in the sense that the absolute number of people who have sufficient understanding of large scale search engines has grown by orders of magnitude in the last 25 years. The challenge is competition. Or rather, the lack thereof. I worked for Google when the last credible challenger threw in the towel and search got pushed closer to being a monopoly / monoculture. At the time Microsoft hadn't gotten their act together. Even as an enthusiastic imbiber of the Google cool-aid, this was deeply worrying. Google distinguished itself because it had to in order to compete. Making suck noticably less and being in tune with the users used to be enough. Because the competition sucked at this. Google no longer has to do that. Jeff Dean doesn't lie awake at night worrying about web search competitors. There was a time when he probably did (if I remember various accounts correctly). I doubt he loses any sleep over it now. Lots of people have tried to do distributed search. Lots and lots. The reason you don't hear about them is that they tend to not produce convincing results. Or in fact any results. In search quality, privacy, fairness. It is much, much harder than you might be tempted to think. And it exists within a field that gets harder every single year since the problem size and complexity keeps growing as the web grows; and grows more complex. If you believe in distributed search I think you should give it a go. Just because everyone has failed before you doesn't necessarily mean it is impossible to do. But it is worth remembering that it is a class of problem that is a lot harder than centralized search. So saying it is the solution is like saying cold fusion is the solution to all our energy needs: it might have been if we knew how to achieve it. But we don't.

streamofdigits 5y ago

Can a single centralized engine / business model ever balance all those conflicts? It feels that the time for decentralized search where entities are both contributing and benefiting from shared search indexes and have the ability to tweak their preferences is, broadly speaking, the space we need to look into?

bborud 5y ago
Centralized vs distributed represents a non-trivial set of technological challenges and differences. Those challenges tended to be poorly understood by anyone who hadn't given the problem of designing a search engine much thought in the past. I don't know if that has improved since I was part of the search engine industry. It probably has in the sense that the absolute number of people who have sufficient understanding of large scale search engines has grown by orders of magnitude in the last 25 years. The challenge is competition. Or rather, the lack thereof. I worked for Google when the last credible challenger threw in the towel and search got pushed closer to being a monopoly / monoculture. At the time Microsoft hadn't gotten their act together. Even as an enthusiastic imbiber of the Google cool-aid, this was deeply worrying. Google distinguished itself because it had to in order to compete. Making suck noticably less and being in tune with the users used to be enough. Because the competition sucked at this. Google no longer has to do that. Jeff Dean doesn't lie awake at night worrying about web search competitors. There was a time when he probably did (if I remember various accounts correctly). I doubt he loses any sleep over it now. Lots of people have tried to do distributed search. Lots and lots. The reason you don't hear about them is that they tend to not produce convincing results. Or in fact any results. In search quality, privacy, fairness. It is much, much harder than you might be tempted to think. And it exists within a field that gets harder every single year since the problem size and complexity keeps growing as the web grows; and grows more complex. If you believe in distributed search I think you should give it a go. Just because everyone has failed before you doesn't necessarily mean it is impossible to do. But it is worth remembering that it is a class of problem that is a lot harder than centralized search. So saying it is the solution is like saying cold fusion is the solution to all our energy needs: it might have been if we knew how to achieve it. But we don't.