6 ms·

Never underestimate the ability of a human being to rationalize. If I was in the Microsoft camp, I'm sure I would also be grasping at straws to explain why it'

Never underestimate the ability of a human being to rationalize. If I was in the Microsoft camp, I'm sure I would also be grasping at straws to explain why it's totally fine for Bing to use Google's search results. It's human nature to rationalize. The bottom line is that Bing's index contains associations that it could never have figured out if Google hadn't figured them out first. How many there are, we cannot know. There's no way around the fact that Bing is piggybacking on the work of Google's search engineers. Is it "good for the customer?" In the short term, it's good for the customer if they can buy $1 bootlegged DVDs. In the long term, it's bad for the consumer if the money goes to bootleggers instead of the people who are doing the actual work. Think I'm exaggerating the effect of just "1 out of 1000 signals?" This argument would be extremely easy to refute. Stop using Google's results. If it really isn't that significant, then why should it be a problem to stop using it? Just turn it off and let everyone observe that the quality is 99.9% as good as it used to be, and avoid any accusation of copying. By refusing to turn it off, Microsoft makes it clear that it is an important part of their index, and that they have no qualms about having an important part of their index ripped off wholesale from their biggest competitor. Maybe it's a smart business move. But if that's the case, spare us the outrage about being called "copyists."

zaidf 16y ago
By that standard, Google should also quit trying to integrate invite-your-fb-friends feature, right? Cuz it's essentially facebook who has figured out how to get massive user signups with their real data and it's google trying to leech. Instead of not trying to integrate fb, Google is accusing fb of not opening up the data. So when it's convenient, you want data opened up. When it's not convenient, you scream "copycat!".
- haberman 16y ago
  Google was freely sharing contact data with Facebook, so there's no hypocrisy here. Contact data is not the same as a search engine.
  - gojomo 16y ago
    Freely sharing the data in the sphere where you're trying to catch up, while zealously guarding the data in the sphere where you've got a massive lead, is the alleged hypocrisy. Hard-won bits is hard-won bits, at a suitably abstract level of analysis.
  - zaidf 16y ago
    Contact data is not the same as a search engine. Elaborate? Bing didn't steal Google's source code. They intercepted some linkstream data with the user's permission.
    - haberman 16y ago
      A search engine performs the task of narrowing 1 trillion URLs into the 10 that are most likely to be what you are looking for. Associating a query with those 10 results takes teams of software engineers and datacenters full of computers. A user could not reasonably do this on their own. Contact data is a list of the people you know. I had that list in my brain before I ever visited Facebook. If I am on different social networking sites, that list is mostly the same, because it is information about me that I already knew. I didn't come to Facebook saying "hey Facebook, find for me the 10 people in the world who would be the best possible friends for me." Instead, I come to Facebook and start telling them the names of my friends. But forget all that for a second. Imagine that contact data and search engines really were the same thing. Google does not access Facebook's social graph, because Facebook has not allowed them to. For Google's behavior to be like Microsoft's Google would have to work around that by using the "clickstream" of Chrome users who are on Facebook. Who knows, maybe Microsoft is doing this already, since they seem to think that "clickstream" is a valid way to mine the internal databases of other companies.
      - zaidf 16y ago
        A search engine performs the task of narrowing 1 trillion URLs into the 10 that are most likely to be what you are looking for. Associating a query with those 10 results takes teams of software engineers and datacenters full of computers. A user could not reasonably do this on their own. Quite irrelevant to the discussion here. A user could suggest specific pages be indexed by Bing, no? If so, why can't a user let Bing automatically listen in on the pages they are visiting and index them if Bing wishes to? That's what Bing did. Contact data is a list of the people you know. I had that list in my brain before I ever visited Facebook. If I am on different social networking sites, that list is mostly the same, because it is information about me that I already knew. Again, pretty irrelevant. If we are talking about data ownership, both your contacts data and your clickstream data belong to you. You can do what you want with it. Some users choose to share it with Bing. That's their prerogative. I don't think even Google disagrees with that. They just don't see the hypocrisy between this and their stance on Facebook. Google does not access Facebook's social graph, because Facebook has not allowed them to. Correct, and yet, Google's stance in the Facebook episode has been that Facebook should open up that data because it does not belong them, it belongs to the users. To be consistent with their stance on Bing, Google should never have created a big fuss about Facebook not opening up the data. Similarly, clickstream data does not belong to Google. It belongs to the users.
joesb 16y ago
I can not agree more with your second paragraph. Yes, Bing toolbar just rely on people clicking links. But without Google doing all the hardwork here, users will never know about such link, that's why they went to Google. And there will not be any of such link to feed to Bing's system.
- seanos 16y ago
  You could also argue that the user is doing the hardwork by going to Google (or any other site, since this is not specific to Google) and deciding which links are relevant by clicking on them. The user has opted share this work they have done with Microsoft (by installing the toolbar and agreeing to the terms) to help them get better results on Bing.
jfager 16y ago
The thing that bothers me about all this drama is that the actual offense Google wants everyone to be so worked up about is that Bing doesn't filter Google from its clickstream data. Bing wrote code that works across the whole web. The whole web includes Google. As a result, Bing gets some info from Google. But they didn't get that info because they copied Google, they got it because they didn't filter Google out - or, said another way, because they ignored Google as someone they needed to special case for clickstream analysis. I don't work in search, but the idea that you're supposed to special-case your competitors when writing general-purpose tools sounds an awful lot like a unilaterally recognized gentleman's agreement. If it's not illegal, and it doesn't hurt end users, why shouldn't it be considered fair game? I also think it's odd that throughout this whole thing, nobody has really noticed that the only possible way Google could have spotted this issue is if they're keeping very close tabs on Bing's search results. It's another arbitrary line that Google seems to have unilaterally drawn: it's clearly fine to monitor your competitors results closely, which presumably is going to have an effect on your own results; it's only out of bounds when that effect is directly measurable.
- nostrademons 16y ago
  I thought part of the point is that whatever Bing is doing doesn't work across the whole web. They need to associate the URL with a query, and most websites don't have queries. It's not just that Bing has recorded a click on Miley Cyrus's webpage; it's that they've done that and associated it with the query [kecgxjpgqoe].
  - jfager 16y ago
    People have made that point, but I don't understand it. 'kecgx...' shows up as a parameter in the url of the Google query. Tons of sites include relevant information in parameterized urls; why is it unexpected that Bing would use that information across the whole web? Other people have said that implies that Bing has to have special Google url-parsing code, but that's not true at all - query parameters in urls are standardized. You would have to have special code to understand the specific semantics of Google's query urls, but there's no reason to think Bing needs or wants parameter semantics, they could easily just be interested in making probabilistic associations.
seanos 16y ago
"The bottom line is that Bing's index contains associations that it could never have figured out if Google hadn't figured them out first." Actually, it is the users who are figuring out what is relevant not Google since a) Bing are collecting data on the links they click regardless of where Google ranks it (could be on the 20th page of Google search results for all Bing care) and b) Bing are collecting this data from all sites, there has been no evidence that Google search results are treated any differently to any other site.
- haberman 16y ago
  > Actually, it is the users who are figuring out what is relevant not Google When you do a Google search, Google chooses ten URLs from over 1 trillion that are in its index. Creating this list takes teams of full-time engineers and data-centers full of servers that crawl the internet constantly. The user spends, on average, less than ten seconds choosing 1 out of those 10 URLs. In a small minority of cases, they'll click to further pages, in which case maybe they were choosing 1 out of 100. So Google narrowed the search down from 1 trillion URLs to 10, and the user narrowed it down from 10 URLs to 1. And yet you think it's the user who did the hard work?
  - seanos 16y ago
    Individually, it negligible. Cumulatively, the time spent by the many thousands (millions?) of users who are clicking links they deem relevant for queries all over the web (Bing are collecting data from all sites) is huge... My point is that the assertion that Google are doing all the work in creating the data used by Bing is clearly false.
    - haberman 16y ago
      > My point is that the assertion that Google are doing all the work in creating the data used by Bing is clearly false. Clearly it's not 100%, but what percentage is it? The user ends up visiting 1 out 1T possible URLs. Google narrowed it down from 1T to 10 URLs (11 orders of magnitude). The user narrowed it down from 10 to 1 URLs (1 order of magnitude). Even if you measure this logarithmically, Google did 91% of the work, and the user did 9%. And the Microsoft people are arguing that it's totally justified because of that 9%. Or you could measure it linearly, and argue that Google did 99.999999999% of the work because that's the percentage of the results they eliminated. Either way, this is an extremely weak justification for taking a result of which 91% came from your direct competitor (when measured in the most charitable way).
sgk284 16y ago
I think you're trying too hard to rationalize Google's claims. With regard to the 1-in-1000 signals, clearly they have that many signals because not all 1000 signals are strong for every query. In Google's case, they "Googlebombed" Bing, but instead of using the "anchor" signal (which most people use to alter Google's results), they chose the "click stream" signal. Google specifically chose scenarios where the other 999 signals weren't being used. If I had a toolbar installed on millions of machines where people opted-in to send me their click activity, I could start a search engine that used only click stream data to rank results. You'd be seeing the ranking effected by all sorts of sites including Facebook, Google, Amazon, Wikipedia, etc... As it should, the ranking would be effected by the click activity that users perform on the most popular websites. Effectively I'd have crowd sourced my ranking. If Google is returning a result for a particular person's name, but Facebook's click stream activity is returning a fan page for that person's name which is clicked on more frequently than the Google result, then the Facebook result would show up in my search engine for that person's name. All Google did here was choose a scenario where every other signal wasn't being used by Bing, effectively turning Bing into the click-stream search engine described above. Furthermore, Google chose a scenario where even within this single signal, they were the only input to it. All Google did was googlebomb Bing. Any well trafficked website could do the same. It's ridiculous the lengths people will go to while trying to discredit Microsoft. Bing has built a generalized system that partially learns rankings through behaviors observed on high-reputation websites. That's it. And no they shouldn't remove Google from the signal because that'd imply they've done something wrong.
- Herring 16y ago
  >As it should, the ranking would be effected by the click activity that users perform on the most popular websites. Effectively I'd have crowd sourced my ranking. Not exactly. Those sites work hard to present relevant links to the crowd. If the sites have competent engineers, your record of clicks cannot add much value -- if just because you only have a subset of all the clicks on that site. So you see google says they're seeing increasing overlap with bing over time. Also see http://news.ycombinator.com/item?id=2184025 http://news.ycombinator.com/item?id=2184025

zaidf 16y ago

By that standard, Google should also quit trying to integrate invite-your-fb-friends feature, right? Cuz it's essentially facebook who has figured out how to get massive user signups with their real data and it's google trying to leech. Instead of not trying to integrate fb, Google is accusing fb of not opening up the data. So when it's convenient, you want data opened up. When it's not convenient, you scream "copycat!".

haberman 16y ago
Google was freely sharing contact data with Facebook, so there's no hypocrisy here. Contact data is not the same as a search engine.
- gojomo 16y ago
  Freely sharing the data in the sphere where you're trying to catch up, while zealously guarding the data in the sphere where you've got a massive lead, is the alleged hypocrisy. Hard-won bits is hard-won bits, at a suitably abstract level of analysis.
- zaidf 16y ago
  Contact data is not the same as a search engine. Elaborate? Bing didn't steal Google's source code. They intercepted some linkstream data with the user's permission.
  - haberman 16y ago
    A search engine performs the task of narrowing 1 trillion URLs into the 10 that are most likely to be what you are looking for. Associating a query with those 10 results takes teams of software engineers and datacenters full of computers. A user could not reasonably do this on their own. Contact data is a list of the people you know. I had that list in my brain before I ever visited Facebook. If I am on different social networking sites, that list is mostly the same, because it is information about me that I already knew. I didn't come to Facebook saying "hey Facebook, find for me the 10 people in the world who would be the best possible friends for me." Instead, I come to Facebook and start telling them the names of my friends. But forget all that for a second. Imagine that contact data and search engines really were the same thing. Google does not access Facebook's social graph, because Facebook has not allowed them to. For Google's behavior to be like Microsoft's Google would have to work around that by using the "clickstream" of Chrome users who are on Facebook. Who knows, maybe Microsoft is doing this already, since they seem to think that "clickstream" is a valid way to mine the internal databases of other companies.
    - zaidf 16y ago
      A search engine performs the task of narrowing 1 trillion URLs into the 10 that are most likely to be what you are looking for. Associating a query with those 10 results takes teams of software engineers and datacenters full of computers. A user could not reasonably do this on their own. Quite irrelevant to the discussion here. A user could suggest specific pages be indexed by Bing, no? If so, why can't a user let Bing automatically listen in on the pages they are visiting and index them if Bing wishes to? That's what Bing did. Contact data is a list of the people you know. I had that list in my brain before I ever visited Facebook. If I am on different social networking sites, that list is mostly the same, because it is information about me that I already knew. Again, pretty irrelevant. If we are talking about data ownership, both your contacts data and your clickstream data belong to you. You can do what you want with it. Some users choose to share it with Bing. That's their prerogative. I don't think even Google disagrees with that. They just don't see the hypocrisy between this and their stance on Facebook. Google does not access Facebook's social graph, because Facebook has not allowed them to. Correct, and yet, Google's stance in the Facebook episode has been that Facebook should open up that data because it does not belong them, it belongs to the users. To be consistent with their stance on Bing, Google should never have created a big fuss about Facebook not opening up the data. Similarly, clickstream data does not belong to Google. It belongs to the users.

joesb 16y ago

I can not agree more with your second paragraph. Yes, Bing toolbar just rely on people clicking links. But without Google doing all the hardwork here, users will never know about such link, that's why they went to Google. And there will not be any of such link to feed to Bing's system.

seanos 16y ago
You could also argue that the user is doing the hardwork by going to Google (or any other site, since this is not specific to Google) and deciding which links are relevant by clicking on them. The user has opted share this work they have done with Microsoft (by installing the toolbar and agreeing to the terms) to help them get better results on Bing.

jfager 16y ago

The thing that bothers me about all this drama is that the actual offense Google wants everyone to be so worked up about is that Bing doesn't filter Google from its clickstream data. Bing wrote code that works across the whole web. The whole web includes Google. As a result, Bing gets some info from Google. But they didn't get that info because they copied Google, they got it because they didn't filter Google out - or, said another way, because they ignored Google as someone they needed to special case for clickstream analysis. I don't work in search, but the idea that you're supposed to special-case your competitors when writing general-purpose tools sounds an awful lot like a unilaterally recognized gentleman's agreement. If it's not illegal, and it doesn't hurt end users, why shouldn't it be considered fair game? I also think it's odd that throughout this whole thing, nobody has really noticed that the only possible way Google could have spotted this issue is if they're keeping very close tabs on Bing's search results. It's another arbitrary line that Google seems to have unilaterally drawn: it's clearly fine to monitor your competitors results closely, which presumably is going to have an effect on your own results; it's only out of bounds when that effect is directly measurable.

nostrademons 16y ago
I thought part of the point is that whatever Bing is doing doesn't work across the whole web. They need to associate the URL with a query, and most websites don't have queries. It's not just that Bing has recorded a click on Miley Cyrus's webpage; it's that they've done that and associated it with the query [kecgxjpgqoe].
- jfager 16y ago
  People have made that point, but I don't understand it. 'kecgx...' shows up as a parameter in the url of the Google query. Tons of sites include relevant information in parameterized urls; why is it unexpected that Bing would use that information across the whole web? Other people have said that implies that Bing has to have special Google url-parsing code, but that's not true at all - query parameters in urls are standardized. You would have to have special code to understand the specific semantics of Google's query urls, but there's no reason to think Bing needs or wants parameter semantics, they could easily just be interested in making probabilistic associations.

seanos 16y ago

"The bottom line is that Bing's index contains associations that it could never have figured out if Google hadn't figured them out first." Actually, it is the users who are figuring out what is relevant not Google since a) Bing are collecting data on the links they click regardless of where Google ranks it (could be on the 20th page of Google search results for all Bing care) and b) Bing are collecting this data from all sites, there has been no evidence that Google search results are treated any differently to any other site.

haberman 16y ago
> Actually, it is the users who are figuring out what is relevant not Google When you do a Google search, Google chooses ten URLs from over 1 trillion that are in its index. Creating this list takes teams of full-time engineers and data-centers full of servers that crawl the internet constantly. The user spends, on average, less than ten seconds choosing 1 out of those 10 URLs. In a small minority of cases, they'll click to further pages, in which case maybe they were choosing 1 out of 100. So Google narrowed the search down from 1 trillion URLs to 10, and the user narrowed it down from 10 URLs to 1. And yet you think it's the user who did the hard work?
- seanos 16y ago
  Individually, it negligible. Cumulatively, the time spent by the many thousands (millions?) of users who are clicking links they deem relevant for queries all over the web (Bing are collecting data from all sites) is huge... My point is that the assertion that Google are doing all the work in creating the data used by Bing is clearly false.
  - haberman 16y ago
    > My point is that the assertion that Google are doing all the work in creating the data used by Bing is clearly false. Clearly it's not 100%, but what percentage is it? The user ends up visiting 1 out 1T possible URLs. Google narrowed it down from 1T to 10 URLs (11 orders of magnitude). The user narrowed it down from 10 to 1 URLs (1 order of magnitude). Even if you measure this logarithmically, Google did 91% of the work, and the user did 9%. And the Microsoft people are arguing that it's totally justified because of that 9%. Or you could measure it linearly, and argue that Google did 99.999999999% of the work because that's the percentage of the results they eliminated. Either way, this is an extremely weak justification for taking a result of which 91% came from your direct competitor (when measured in the most charitable way).

sgk284 16y ago

I think you're trying too hard to rationalize Google's claims. With regard to the 1-in-1000 signals, clearly they have that many signals because not all 1000 signals are strong for every query. In Google's case, they "Googlebombed" Bing, but instead of using the "anchor" signal (which most people use to alter Google's results), they chose the "click stream" signal. Google specifically chose scenarios where the other 999 signals weren't being used. If I had a toolbar installed on millions of machines where people opted-in to send me their click activity, I could start a search engine that used only click stream data to rank results. You'd be seeing the ranking effected by all sorts of sites including Facebook, Google, Amazon, Wikipedia, etc... As it should, the ranking would be effected by the click activity that users perform on the most popular websites. Effectively I'd have crowd sourced my ranking. If Google is returning a result for a particular person's name, but Facebook's click stream activity is returning a fan page for that person's name which is clicked on more frequently than the Google result, then the Facebook result would show up in my search engine for that person's name. All Google did here was choose a scenario where every other signal wasn't being used by Bing, effectively turning Bing into the click-stream search engine described above. Furthermore, Google chose a scenario where even within this single signal, they were the only input to it. All Google did was googlebomb Bing. Any well trafficked website could do the same. It's ridiculous the lengths people will go to while trying to discredit Microsoft. Bing has built a generalized system that partially learns rankings through behaviors observed on high-reputation websites. That's it. And no they shouldn't remove Google from the signal because that'd imply they've done something wrong.

Herring 16y ago
>As it should, the ranking would be effected by the click activity that users perform on the most popular websites. Effectively I'd have crowd sourced my ranking. Not exactly. Those sites work hard to present relevant links to the crowd. If the sites have competent engineers, your record of clicks cannot add much value -- if just because you only have a subset of all the clicks on that site. So you see google says they're seeing increasing overlap with bing over time. Also see http://news.ycombinator.com/item?id=2184025 http://news.ycombinator.com/item?id=2184025