3 ms·

>Its because the second order effect of a click implies relevance, just as a link does. There are two pieces of information in a click, one provided by google,

>Its because the second order effect of a click implies relevance, just as a link does. There are two pieces of information in a click, one provided by google, and the other provided by the user. Google says, "'foo.com' is a good result for [foo]." The user says, "yup." Which is providing more information here? Everyone at google is happy to admit that if the internet didn't exist, we would have a very hard time ranking it. Is bing ready to admit that if google didn't exist, they'd have a hard time ranking the internet?

kenjackson 16y ago
Your first statement is wrong. The user provides at least two pieces of information. 1) The search query (which you attmpted to hide in Google's information) 2) A URL they retrieve from Google Google provides one piece: 1) An ordered list By FAR the most important piece of information is the search query. In fact, given a choice between having the search query or an arbitrary ordered list, I'm sure Bing would value the search query the most. Clickstream data from Google is almost certainly but one piece of clickstream data (even if you are special-cased). Just as Microsoft.com is just one page you index. Now you wouldn't say that you'd have time ranking the internet if Microsoft.com didn't exist would you? Of course not. And I think this is where Google is going astray. It's fine to call a spade a spade, but your overreaching. Trying to argue that MS couldn't index w/o Google is simply absurd. And if Matt was trying to be diplomatic, you're certainly not being by implying that MS would have a hard time indexing w/o Google.
- moultano 16y ago
  >Google provides one piece: 1) An ordered list That's not an accurate view of how search works. The user provides the query, but the results that are returned don't need to have anything obvious to do with the query. The terms might not be on the page, and might not be found in any data remotely associated with the page. We don't just intersect posting lists here. Furthermore, with suggest and instant, often the query itself is provided by google.
  - kenjackson 16y ago
    I didn't say the ordered list was obvious. But it is an ordered list (at least to the user -- it may not be ordered all the way down or partially so). Fair enough point about suggest and instant. In those cases Google is adding value to the query, but I think we'd both agree that it is incrementally so. Instant generally just helps you get to the eventual query faster. Spell checking does add value, but its a +epsilon. In either case, your right, we should add this to our function. In any case I think my point still holds. The query is the most important aspect. The browsed page, for indexing purposes is the second most important aspect. The association between the query and the URL is the least important aspect. IOW, the least value for Bing is the search algorithm. The query and page to add to the index (as a browsed page) are the most important.
    - moultano 16y ago
      >The association between the query and the URL is the least important aspect. I don't understand why you believe this. This is the entire basis for why search is hard. It is the hard problem that search tries to solve. Many companies have spent in aggregate, billions of dollars in R&D trying to solve this, and all but a few have folded. It's an "AI-complete" problem in that solving it perfectly would be a sufficient demonstration of strong AI. It's the whole reason why Google exists.
      - kenjackson 16y ago
        The reason is that with the URL and the query, there's a decent chance you can derive the association. Now the reason I say this is that to my best approximation the key difference that makes Google better than Bing (when it is) is the index size, and not the search algorithm. So the two key pieces of value are: A) queries that people don't do on your site, which would have generated poor relevance B) Pages that are actually browsed that you don't have indexed or up to date. Now don't get me wrong. There is value in the association, but I think I'd capture 80% of the value with the above. Now as you point out, these lists are often non-obvious, but Bing does a near equal job of creating the lists. And if you give them the search terms where they have gaps. And fill the index, then I think the gap closes most of the way. And lets be clear, because they clicked the link doesn't mean its a good link. See ehow.com or expertsexchange. But it does have some value. And frankly I think MS would have been willing to let it go if Google had gone straight to MS with it and said, "We have this data. Even though its not unethical we can spin it to the media to make it look bad. Kill the association." MS probably would have as the net win isn't that huge. Now the PR loss in pulling it would be worse than the PR loss in keeping it (with no integrity loss, since they [and I] think it is perfectly ethical).

kenjackson 16y ago

Your first statement is wrong. The user provides at least two pieces of information. 1) The search query (which you attmpted to hide in Google's information) 2) A URL they retrieve from Google Google provides one piece: 1) An ordered list By FAR the most important piece of information is the search query. In fact, given a choice between having the search query or an arbitrary ordered list, I'm sure Bing would value the search query the most. Clickstream data from Google is almost certainly but one piece of clickstream data (even if you are special-cased). Just as Microsoft.com is just one page you index. Now you wouldn't say that you'd have time ranking the internet if Microsoft.com didn't exist would you? Of course not. And I think this is where Google is going astray. It's fine to call a spade a spade, but your overreaching. Trying to argue that MS couldn't index w/o Google is simply absurd. And if Matt was trying to be diplomatic, you're certainly not being by implying that MS would have a hard time indexing w/o Google.

moultano 16y ago
>Google provides one piece: 1) An ordered list That's not an accurate view of how search works. The user provides the query, but the results that are returned don't need to have anything obvious to do with the query. The terms might not be on the page, and might not be found in any data remotely associated with the page. We don't just intersect posting lists here. Furthermore, with suggest and instant, often the query itself is provided by google.
- kenjackson 16y ago
  I didn't say the ordered list was obvious. But it is an ordered list (at least to the user -- it may not be ordered all the way down or partially so). Fair enough point about suggest and instant. In those cases Google is adding value to the query, but I think we'd both agree that it is incrementally so. Instant generally just helps you get to the eventual query faster. Spell checking does add value, but its a +epsilon. In either case, your right, we should add this to our function. In any case I think my point still holds. The query is the most important aspect. The browsed page, for indexing purposes is the second most important aspect. The association between the query and the URL is the least important aspect. IOW, the least value for Bing is the search algorithm. The query and page to add to the index (as a browsed page) are the most important.
  - moultano 16y ago
    >The association between the query and the URL is the least important aspect. I don't understand why you believe this. This is the entire basis for why search is hard. It is the hard problem that search tries to solve. Many companies have spent in aggregate, billions of dollars in R&D trying to solve this, and all but a few have folded. It's an "AI-complete" problem in that solving it perfectly would be a sufficient demonstration of strong AI. It's the whole reason why Google exists.
    - kenjackson 16y ago
      The reason is that with the URL and the query, there's a decent chance you can derive the association. Now the reason I say this is that to my best approximation the key difference that makes Google better than Bing (when it is) is the index size, and not the search algorithm. So the two key pieces of value are: A) queries that people don't do on your site, which would have generated poor relevance B) Pages that are actually browsed that you don't have indexed or up to date. Now don't get me wrong. There is value in the association, but I think I'd capture 80% of the value with the above. Now as you point out, these lists are often non-obvious, but Bing does a near equal job of creating the lists. And if you give them the search terms where they have gaps. And fill the index, then I think the gap closes most of the way. And lets be clear, because they clicked the link doesn't mean its a good link. See ehow.com or expertsexchange. But it does have some value. And frankly I think MS would have been willing to let it go if Google had gone straight to MS with it and said, "We have this data. Even though its not unethical we can spin it to the media to make it look bad. Kill the association." MS probably would have as the net win isn't that huge. Now the PR loss in pulling it would be worse than the PR loss in keeping it (with no integrity loss, since they [and I] think it is perfectly ethical).