5 ms·

> They should focus on trying to be innovative again, that was the Google I respected. I work in search quality at Google. I'm busting my ass every day working

> They should focus on trying to be innovative again, that was the Google I respected. I work in search quality at Google. I'm busting my ass every day working on fundamental reimaginings of how results get ranked. I'm going to keep doing that regardless of what bing does, because it makes the world a better place, and it's fun. But, suppose the stuff I'm working on works out, and tomorrow Google shows up with a wholly new set of awesome results. This is very possible, there is a ton of headroom left in search quality, I've seen the experiments myself. Then after a few weeks of sniffing clicks, Bing comes up with the same set of revolutionary results, but they have no idea how they got there, they have no idea what the evidence is to rank them there, all they know is that people like those results on Google. Is this fair? Is this ethical? Is this even legal? People in this whole debate have the idea that the user is creating this association between the result and the query, like the user searched through the whole web and came back saying "hey microsoft, check out this great result for this query! I found it! Isn't is awesome?" They aren't. Users click on whatever results you put in front of them, generally starting with the top and working their way down. Ranking results is not a science with some objective optimal conclusion. Ranking results is fundamentally subjective, and while data-driven, is ultimately an opinion. The user does have some discriminating power in this whole feedback loop, but it's miniscule compared to Google figuring out how to show them that result in the first place. Bing is taking the closest proxy that they can practically acquire for Google's opinion, and using it directly in their ranking.

jdp23 16y ago
I remember back when people hand-crafted directories for the web. They'd put a lot of work into updating their directory and then in just a couple of days an automated spider would look at their site and a search engine would use their link structure to point everybody directly to the best pages without having to go through the directory. The search engine had no idea of what evidence the directory used. Was it fair? Was it ethical? Was it legal?
- moultano 16y ago
  robots.txt
  - jdp23 16y ago
    robots.txt is a good example of something that was introduced to make the new search engine technology more ethical but I'm not sure what point you are trying to make. If you're saying that we need a new robots.txt option saying "don't use clickstream data to help people find this page", I don't disagree; I'm not sure how many sites would take advantage of it though.
    - moultano 16y ago
      You're addressing the letter of the "law" not the spirit. Do we really need a new formal standard to indicate that this is unethical? To me it's pretty plain from the standards the web has already agreed upon.
      - jdp23 16y ago
        To others it's just as plain that Microsoft's behavior is ethical. I'm still waiting to hear more before making up my mind. I don't blame you for being upset and frustrated, and can see why you think it's unethical. But a lot of people see it differently. It would be great for Google and Microsoft to both come clean about all the different factors they use in their search but I'm not holding my breath.
      - greendestiny 16y ago
        Absolutely a new standard is needed. This is the first time any major player has suggested that data from a users clickstream should be covered by robots.txt or anything like it. The comparison to a web directory is perfect - in the absence of a robots.txt Google and other crawlers think the data is absolutely fair game. There isn't yet a corresponding analogy for clickstream data so its fair game. Are you sure all of Google's tools respect robots.txt for passive analysis of user initiated actions on other websites?
        16y ago
tygorius 16y ago
Is this fair? Is this ethical? Is this even legal? Oh please. At worst, Microsoft's customers are sharing results of their browsing activities, based on information you gave them. How in the world could that be illegal? Your team is really coming across as whiny on this issue, which seems to me precisely the wrong tactic to take in response to Bing's increasing market share. It makes your team look defensive, as if your advantage is slipping away and this sort of theatrical display is all you've got in response.
- rayval 16y ago
  I guess it's in the eye of the beholder. Others interpret this situation as Bing being unable to compete effectively except by looking over Google's shoulder. And then denying it when called out with clear-cut evidence.
davidu 16y ago
Keep doing what you are doing and don't take what I wrote too personally. It's obvious that there are a lot of amazingly talented individuals working hard at Google. IMHO - One of your colleagues should have stopped that blog post from being put up.
kenjackson 16y ago
Then after a few weeks of sniffing clicks, Bing comes up with the same set of revolutionary results, but they have no idea how they got there, they have no idea what the evidence is to rank them there, all they know is that people like those results on Google. But this whole "no idea how they got there" is kind of bogus. The whole notion of link analysis is to infer from a link that a page is important. Now they're infering from a click that page is important. The relative importance of that page is a function of the search query, like the relative importance of a link is a function of the page from which it originated. Link analysis and click throughs are both second order effects, right? If CNN.com starts linking to a page, you don't really know why that page is important, except for the fact that CNN.com now points to it. So lets be clear MS knows how it got there as much as link analysis does. Its because the second order effect of a click implies relevance, just as a link does.
- pessimist 16y ago
  The click is stronger than the link, because MS is including the query associated with the click.
- moultano 16y ago
  >Its because the second order effect of a click implies relevance, just as a link does. There are two pieces of information in a click, one provided by google, and the other provided by the user. Google says, "'foo.com' is a good result for [foo]." The user says, "yup." Which is providing more information here? Everyone at google is happy to admit that if the internet didn't exist, we would have a very hard time ranking it. Is bing ready to admit that if google didn't exist, they'd have a hard time ranking the internet?
  - kenjackson 16y ago
    Your first statement is wrong. The user provides at least two pieces of information. 1) The search query (which you attmpted to hide in Google's information) 2) A URL they retrieve from Google Google provides one piece: 1) An ordered list By FAR the most important piece of information is the search query. In fact, given a choice between having the search query or an arbitrary ordered list, I'm sure Bing would value the search query the most. Clickstream data from Google is almost certainly but one piece of clickstream data (even if you are special-cased). Just as Microsoft.com is just one page you index. Now you wouldn't say that you'd have time ranking the internet if Microsoft.com didn't exist would you? Of course not. And I think this is where Google is going astray. It's fine to call a spade a spade, but your overreaching. Trying to argue that MS couldn't index w/o Google is simply absurd. And if Matt was trying to be diplomatic, you're certainly not being by implying that MS would have a hard time indexing w/o Google.
lmkg 16y ago
You have a valid point that the click-through measures more of what Google puts out rather than what users find useful. But Bing has access not just to the click-through rate but also the rest of the clickstream. IF (and that's an all-caps if) the value of the click is weighted by engagement metrics on the resulting page, the association is more closely related to the value the user finds in the page rather than the results that Google returned. Would you have a problem with Bing doing that? As a user, and not a developer, I rather like it because it's a much more direct metric of whether a link is useful or not.
- deleted 16y ago
  [deleted]
- moultano 16y ago
  They are free to do that on their own search results. If they can't figure out how to rank the document in the first place, they shouldn't be getting data about the association from google.
nostrademons 16y ago
If they don't understand the results and how they got there, how will they improve them?
DenisM 16y ago
You have computed a relationship between FOO and BAR. For as long as you keep it to yourself, this ephemeral relationship is a secret, yours to keep. Once you tell the world that FOO and BAR are related, and the whole world looks at it and says "Yay, it IS related!", the relationship stops being ephemeral and becomes actual, reflected in actions of the users. You no loner own that relationship. You can't claim ownership of facts, even if you discovered them first, or created them into existence.
- moultano 16y ago
  >You can't claim ownership of facts, even if you discovered them first, or created them into existence. It's not as straightforward as all that. Suppose you write a book. The existence of that book is a fact. The words that are in that book are now a fact. Now, I create a book that says, "The following book was written by DennisM, this is a list of the words that are in that book, in order" and then precede to duplicate your book. Those are the words that are in the book, it's a fact, no one can deny that it's a fact. That's not defensible, even though I'm just stating a fact. We're in a somewhat grey area here, dealing with ethical issues that have never been clearly dealt with before. However, I feel that a general principle of the web applies: If I create some data, and don't at least implicitly give you permission to use it, it's unethical for you to use it. This is the central idea behind copyright, behind robots.txt, behind plagiarism, and even behind privacy.
  - tygorius 16y ago
    Actually, it is as straightforward as all that. What's alleged is that someone searches for DenisM's book on Google, and when they click on one of the results, just that piece of data is forwarded to Microsoft. Not a scraping of all of Google's search results and nothing like the contents of the book. As to "the central idea of copyright", bear in mind that in addition to not being able to copyright ideas or data, there's also the notion of "fair use".
    - moultano 16y ago
      >nothing like the contents of the book. Explain this to me. In terms of raw bytes it's probably more data than every book that has ever been written. It was certainly more expensive to write than any particular book. What is the important difference here? To make it a little more concrete, suppose someone was doing the same thing with google's streetview data. Now, it's an empirical fact that if you go to such-and-such an address, and look around, it looks like the pictures that google took. Does that mean it's ok to use those pictures, just because their resemblance to reality is factual?

jdp23 16y ago

I remember back when people hand-crafted directories for the web. They'd put a lot of work into updating their directory and then in just a couple of days an automated spider would look at their site and a search engine would use their link structure to point everybody directly to the best pages without having to go through the directory. The search engine had no idea of what evidence the directory used. Was it fair? Was it ethical? Was it legal?

moultano 16y ago
robots.txt
- jdp23 16y ago
  robots.txt is a good example of something that was introduced to make the new search engine technology more ethical but I'm not sure what point you are trying to make. If you're saying that we need a new robots.txt option saying "don't use clickstream data to help people find this page", I don't disagree; I'm not sure how many sites would take advantage of it though.
  - moultano 16y ago
    You're addressing the letter of the "law" not the spirit. Do we really need a new formal standard to indicate that this is unethical? To me it's pretty plain from the standards the web has already agreed upon.
    - jdp23 16y ago
      To others it's just as plain that Microsoft's behavior is ethical. I'm still waiting to hear more before making up my mind. I don't blame you for being upset and frustrated, and can see why you think it's unethical. But a lot of people see it differently. It would be great for Google and Microsoft to both come clean about all the different factors they use in their search but I'm not holding my breath.
    - greendestiny 16y ago
      Absolutely a new standard is needed. This is the first time any major player has suggested that data from a users clickstream should be covered by robots.txt or anything like it. The comparison to a web directory is perfect - in the absence of a robots.txt Google and other crawlers think the data is absolutely fair game. There isn't yet a corresponding analogy for clickstream data so its fair game. Are you sure all of Google's tools respect robots.txt for passive analysis of user initiated actions on other websites?
      - 16y ago

tygorius 16y ago

Is this fair? Is this ethical? Is this even legal? Oh please. At worst, Microsoft's customers are sharing results of their browsing activities, based on information you gave them. How in the world could that be illegal? Your team is really coming across as whiny on this issue, which seems to me precisely the wrong tactic to take in response to Bing's increasing market share. It makes your team look defensive, as if your advantage is slipping away and this sort of theatrical display is all you've got in response.

rayval 16y ago
I guess it's in the eye of the beholder. Others interpret this situation as Bing being unable to compete effectively except by looking over Google's shoulder. And then denying it when called out with clear-cut evidence.

davidu 16y ago

Keep doing what you are doing and don't take what I wrote too personally. It's obvious that there are a lot of amazingly talented individuals working hard at Google. IMHO - One of your colleagues should have stopped that blog post from being put up.

kenjackson 16y ago

Then after a few weeks of sniffing clicks, Bing comes up with the same set of revolutionary results, but they have no idea how they got there, they have no idea what the evidence is to rank them there, all they know is that people like those results on Google. But this whole "no idea how they got there" is kind of bogus. The whole notion of link analysis is to infer from a link that a page is important. Now they're infering from a click that page is important. The relative importance of that page is a function of the search query, like the relative importance of a link is a function of the page from which it originated. Link analysis and click throughs are both second order effects, right? If CNN.com starts linking to a page, you don't really know why that page is important, except for the fact that CNN.com now points to it. So lets be clear MS knows how it got there as much as link analysis does. Its because the second order effect of a click implies relevance, just as a link does.

pessimist 16y ago
The click is stronger than the link, because MS is including the query associated with the click.
moultano 16y ago
>Its because the second order effect of a click implies relevance, just as a link does. There are two pieces of information in a click, one provided by google, and the other provided by the user. Google says, "'foo.com' is a good result for [foo]." The user says, "yup." Which is providing more information here? Everyone at google is happy to admit that if the internet didn't exist, we would have a very hard time ranking it. Is bing ready to admit that if google didn't exist, they'd have a hard time ranking the internet?
- kenjackson 16y ago
  Your first statement is wrong. The user provides at least two pieces of information. 1) The search query (which you attmpted to hide in Google's information) 2) A URL they retrieve from Google Google provides one piece: 1) An ordered list By FAR the most important piece of information is the search query. In fact, given a choice between having the search query or an arbitrary ordered list, I'm sure Bing would value the search query the most. Clickstream data from Google is almost certainly but one piece of clickstream data (even if you are special-cased). Just as Microsoft.com is just one page you index. Now you wouldn't say that you'd have time ranking the internet if Microsoft.com didn't exist would you? Of course not. And I think this is where Google is going astray. It's fine to call a spade a spade, but your overreaching. Trying to argue that MS couldn't index w/o Google is simply absurd. And if Matt was trying to be diplomatic, you're certainly not being by implying that MS would have a hard time indexing w/o Google.

lmkg 16y ago

You have a valid point that the click-through measures more of what Google puts out rather than what users find useful. But Bing has access not just to the click-through rate but also the rest of the clickstream. IF (and that's an all-caps if) the value of the click is weighted by engagement metrics on the resulting page, the association is more closely related to the value the user finds in the page rather than the results that Google returned. Would you have a problem with Bing doing that? As a user, and not a developer, I rather like it because it's a much more direct metric of whether a link is useful or not.

deleted 16y ago
[deleted]
moultano 16y ago
They are free to do that on their own search results. If they can't figure out how to rank the document in the first place, they shouldn't be getting data about the association from google.

nostrademons 16y ago

If they don't understand the results and how they got there, how will they improve them?

DenisM 16y ago

You have computed a relationship between FOO and BAR. For as long as you keep it to yourself, this ephemeral relationship is a secret, yours to keep. Once you tell the world that FOO and BAR are related, and the whole world looks at it and says "Yay, it IS related!", the relationship stops being ephemeral and becomes actual, reflected in actions of the users. You no loner own that relationship. You can't claim ownership of facts, even if you discovered them first, or created them into existence.

moultano 16y ago
>You can't claim ownership of facts, even if you discovered them first, or created them into existence. It's not as straightforward as all that. Suppose you write a book. The existence of that book is a fact. The words that are in that book are now a fact. Now, I create a book that says, "The following book was written by DennisM, this is a list of the words that are in that book, in order" and then precede to duplicate your book. Those are the words that are in the book, it's a fact, no one can deny that it's a fact. That's not defensible, even though I'm just stating a fact. We're in a somewhat grey area here, dealing with ethical issues that have never been clearly dealt with before. However, I feel that a general principle of the web applies: If I create some data, and don't at least implicitly give you permission to use it, it's unethical for you to use it. This is the central idea behind copyright, behind robots.txt, behind plagiarism, and even behind privacy.
- tygorius 16y ago
  Actually, it is as straightforward as all that. What's alleged is that someone searches for DenisM's book on Google, and when they click on one of the results, just that piece of data is forwarded to Microsoft. Not a scraping of all of Google's search results and nothing like the contents of the book. As to "the central idea of copyright", bear in mind that in addition to not being able to copyright ideas or data, there's also the notion of "fair use".
  - moultano 16y ago
    >nothing like the contents of the book. Explain this to me. In terms of raw bytes it's probably more data than every book that has ever been written. It was certainly more expensive to write than any particular book. What is the important difference here? To make it a little more concrete, suppose someone was doing the same thing with google's streetview data. Now, it's an empirical fact that if you go to such-and-such an address, and look around, it looks like the pictures that google took. Does that mean it's ok to use those pictures, just because their resemblance to reality is factual?