6 ms·

If it were just click data, how would they get the terms? They're either parsing the query out of the url, or violating robots.txt to fetch the result page, al

If it were just click data, how would they get the terms? They're either parsing the query out of the url, or violating robots.txt to fetch the result page, almost certainly the former. This seems like a pretty clear indication that they've special-cased clicks from google. It's theoretically plausible that they are treating all query parameters the same for all sites, but very unlikely given how much noise that would introduce into their results. Even so, they would have to know that most clicks with meaningful query parameters come from Google. This isn't something that's going to happen by accident.

brudgers 16y ago
Twenty Google engineers were charged with getting Bing to hit the honeypots and given several weeks to do so. Balancing the oddity of the query terms with the level of resources Google threw into the operation, I'm not sure if 7/100 is impressive SEO or unimpressive. What I do know is that the original suspicious result for "torsoraphy" is returned by Wikimedia's search algorithm. http://en.wikipedia.org/wiki/Special:Search?search=torsoraphy http://en.wikipedia.org/wiki/Special:Search?search=torsoraph...
- Tiomaidh 16y ago
  No, Wikipedia admin Nihiltres intentionally created the "torsoraphy" page (at 3:03 today, no less) and made it redirect to "Tarsorrhaphy". It's not the search algorithm being clever. http://en.wikipedia.org/w/index.php?title=Torsoraphy&action=history http://en.wikipedia.org/w/index.php?title=Torsoraphy&act...
bigiain 16y ago
"a pretty clear indication that they've special-cased clicks from google" - just like pretty much every piece of web analytic software since awstats has done for over 10 years, right? To imagine that _any_ half serious search engine isn't special-casing any referrer information it can get it's hands on from the top half of http://www.alexa.com/topsites http://www.alexa.com/topsites is surely naive?
nikster 16y ago
Well, no. Imagine I am the Bing toolbar and I am spying on my users. The Google search result for fake word zxczxczxczxczx is then a web page with a single link, and the user I am observing is clicking that link. So I am Bing, and I make a connection: zxczxczxczxczx (which appears on this web page) is related to said link. Since the word is artificial and doesn't appear anywhere else, it's eventually going to produce that page as a search result. So I think the author is right - what Google has done is they have proven that Bing toolbar tracks what users are doing, and sends the results back to Bing to improve their search. They have not proven that Bing treats google.com differently from any other page. For that they should have seeded 100 random web pages from different domains with artificial words, and have these pages contain a link which the user with Bing toolbar installed then clicks. I bet that it would yield the same result as the Google test, e.g. those fake words would get matched to the linked web pages.
- barrkel 16y ago
  zxczxczxczxczx isn't on the web page. It's on the search results page, but that page isn't for crawling, by robots.txt. Either MS is ignoring robots.txt, or they're parsing the URLs.
  - bigiain 16y ago
    From my reading of the article, zxczxczxczxczx isn't in the search results page either, it was special cased at google to display the honeypot page for that specific query. But is Bing "parsing the url" any different to what Google's doing when I go into Google Analytics -> Traffic Sources -> Search Engines, and select Keyword in the second dropdown (after source)? Google are clearly showing me the search terms parsed out of the referrer urls from people who found my site in Bing. I think this is perhaps not obvious behaviour to the general public, but surely pretty much anyone who goes to the trouble of installing the Bing (or Google) toolbar has worked out for themselves that they're choosing to send data like this to Microsoft (and Google), and that it'll get used to "improve search" if it's found to be useful for that?
    - Terretta 16y ago
user24 16y ago
> parsing the query out of the url Or parsing any words found in the URL. Not unreasonable to believe.
- moultano 16y ago
  If that were the explanation, I would think they would come out and say it. (Maybe they will shortly.) Even so, I find it hard to believe that they wouldn't notice that most of the benefit of the technique comes from recreating google results, at which point we're back at the original ethical question of whether that's ok.
  - dunham 16y ago
    most of the benefit of the technique comes from recreating google results I'm not 100% sure this is the case. Consider that an user wants a chicken enchilada recipe. 1. They go to a site that they trust for high quality data. (I like food&wine, saveur, epicurious, rick bayless for mexican, ...) 2. They search for "chicken enchilada" 3. They select the most appealing result (not necessarily the first one), based on author, snippet, rating, photo, etc. Domain knowledge. If you can associate (via the referrer) "chicken enchilada" with that page, you've encapsulated a lot of information - manual selection of site, manual selection of a page within that site. It's potentially useful (especially for long tail - people who have to go directly to sites because search engines fail them). Is there too much noise? Perhaps - you'd really have to look at the raw data to find out. Maybe you only want to look at "q=", "search=", and whatever phpBB, etc use. Maybe the signal that rises out of that noise is valuable. One big downside is that the results would be biased towards the preferences of people who install toolbars. :) The other downside is that SEOs could game the system by feeding MS bad data. (True if they're only looking at google data, too.)
- forgotusername 16y ago
  Not only does it not seem unreasonable, but for a company like Google, I find it incredibly hard to believe that at some point they wouldn't have tried it themselves, at least as an experiment.
othermaciej 16y ago
I think their terms of use say they may pass along the contents of form fields. On a Google search results page, your search query is sitting in a text field.
- moultano 16y ago
  Suppose it uses all text fields on the web without treating google specially. Being as charitable as possible here, I'm willing to believe that some well-meaning engineer coded this up without special casing google. Tested it out, found that it worked amazingly well, and then launched it. This seems unlikely, but possible. However, somewhere along the line someone must have known that the biggest benefit of this signal was recreating Google results. I'm not willing to believe that no one figured this out even if it wasn't the initial intent. At which point there's an ethical dilemma. At Google, a system like this wouldn't launch. Regardless of the mechanism, I don't believe that nobody at bing knows that this is what was going on. Maybe it's a cynical attempt to get around robots.txt. Maybe it's an honest mistake that gradually became a dishonest mistake, but I'm not willing to believe that they are oblivious.
  - greendestiny 16y ago
    There is a huge difference between recreating Google results and incorporating a Google honeypot link into their results through legitimate means. One link isn't a search result, a search result is an ordered list of results. If its whole search results then Bing's got some questions to answer and deserves some bad press. Otherwise, not so much.
  - othermaciej 16y ago
    Really? I doubt replicating Google long tail queries is the biggest benefit. User clicks seem like a useful signal of relevance for the same reason links from other sites are a useful signal of relevance.
    - moultano 16y ago
      Long tail queries are the ones which Google could easily demonstrate the effect without any confounding variables. I'd imagine this is affecting all of their ranking, as much as their own click data weighted by volume.
ddemchuk 16y ago
Considering they're a search engine, whose job is to crawl the internet and determine the main keywords of any random page on the internet, I would imagine they have text parsing abilities that make it possible to infer the topic of any arbitrary page.
toolate 16y ago
IE (with certain settings on) is sending page data back to Microsoft. If it sends the URL, title and referrer back then the following session is pretty easy to reverse engineer. 1. URL: "test - Google" Title: http://www.google.com/search?q=test 2. URL: http://test.org.us Title: "Test" Referrer: http://www.google.com/search?q=test Time: 2 secs 2. URL: http://en.wikipedia.org/wiki/Test_cricket Title: "Test cricket - Wikipedia, the free encyclopedia" Referrer: http://www.google.com/search?q=test Time: 249 secs It's really just an extension of page rank by seeing what links are being clicked on and not just which links exist. Whether MS should be capturing this data under false pretenses is another issue.
- eddieplan9 16y ago
  If this is the case, then it's rather easy to stop Microsoft from doing this. Just use POST instead of GET in the search page if you detect the browser is IE8. The referrer will always be the generic http://www.google.com/search http://www.google.com/search with no search term information.
  - rjvir 16y ago
    Using POST instead of GET is not a good idea for a search results page. Most likely, the user would have to click through a dialog box ("are you sure you want to resubmit this form") every time the browser back button is used to return to the search results. Even if Google used redirects to circumvent this problem, searches won't be saved in the browser's history, which is a bit inconvenient. Changing from POST to GET in IE8 would stop Microsoft from mining data in the short term, but would drastically decrease the UX of Google for a large portion of its users who could very easily switch to Bing.
    - eftpotrm 16y ago
      OK, so for IE8 users return a single page AJAX app instead as a variant of what's already done with Instant. Still no referrers but no POST warning messages (which, BTW, have to be one of the most annoying things about ASP.Net web forms - postback was a boneheaded design decision from someone who didn't understand HTTP.)
ac2u 16y ago
"They're either parsing the query out of the url, or violating robots.txt to fetch the result page, almost certainly the former." Where's your evidence for any of that? There's another way. Store the input of any Text Box which is immediately followed by a form submit,(search term), and then store the href of the subsequent click (the result which the user finds helpful). The above is a way of handling it (naive and abstract) which doesn't target Google, and in fact would work for insite searches as well, or any search engine. Since you've already made up your mind that your two ways were the only way of doing things, what's your opinion given the above?
- sonoffett 16y ago
  I'm not sure why you are being voted down, the situation could be this. Also, Bing toolbar could simply be collecting tuples "(search_string_entered_in_toolbar, subsequent_href_clicked)" regardless of the search engine the user has specified and phoning these home... this seems like the easiest implementation (no parsing nonsense) and does not target google in particular.
bambax 16y ago
> If it were just click data, how would they get the terms? Exactly. It's not "click data" at all. It is monitoring user behavior on search engines, using both the clicks and the queries. Maybe it's not monitoring just Google as a search engine (although we have no proof of that yet: it seems it's just watching Google) but given Google's market share in search it doesn't make much difference. From the article: > I don’t even work in search and I could spot the real situation This sentence is at the same time arrogant and funny. He doesn't work in search, but he's certain he's spotted "the real situation". How did he fact check it, besides asking himself if he was correct and answering "yes, obviously I'm right. I'm always right -- and I don't even know anything! I amaze myself."
- user24 16y ago
  > certain he's spotted "the real situation". How did he fact check it By reading the official MS blog post[1], watching the Farsight conference live, and being confident in my own judgements based on the observed evidence. No other conclusion is supported by the evidence - you say "It is monitoring user behavior on search engines" - where's the evidence that it's exclusively search engines they're monitoring? Let alone exclusively Google? There has been no evidence yet presented. Therefore all we can conclude from Google's sting is that Bing use URL/click data. Which is exactly what I said in the original thread[2], in my blog post, and lo and behold it's what MS later said. That's how I know it's the real situation. [1] like where Harry says "A small piece of that is clickstream data we get from some of our customers, who opt-in ... To be clear, we learn from all of our customers". [2] http://news.ycombinator.com/item?id=2165947 http://news.ycombinator.com/item?id=2165947
  - Terretta 16y ago
    You checked it against the assertion of the most interested party? It's not "click" data, it's the correlation between search term and SERP. That someone visited such and such a page after a click isn't the issue. That someone Googled for a unique term before clicking is at issue. Grandparent: http://news.ycombinator.com/item?id=2168377 http://news.ycombinator.com/item?id=2168377 Googling with Bing: http://www.collegehumor.com/video:1915736 http://www.collegehumor.com/video:1915736

brudgers 16y ago

Twenty Google engineers were charged with getting Bing to hit the honeypots and given several weeks to do so. Balancing the oddity of the query terms with the level of resources Google threw into the operation, I'm not sure if 7/100 is impressive SEO or unimpressive. What I do know is that the original suspicious result for "torsoraphy" is returned by Wikimedia's search algorithm. http://en.wikipedia.org/wiki/Special:Search?search=torsoraphy http://en.wikipedia.org/wiki/Special:Search?search=torsoraph...

Tiomaidh 16y ago
No, Wikipedia admin Nihiltres intentionally created the "torsoraphy" page (at 3:03 today, no less) and made it redirect to "Tarsorrhaphy". It's not the search algorithm being clever. http://en.wikipedia.org/w/index.php?title=Torsoraphy&action=history http://en.wikipedia.org/w/index.php?title=Torsoraphy&act...

bigiain 16y ago

"a pretty clear indication that they've special-cased clicks from google" - just like pretty much every piece of web analytic software since awstats has done for over 10 years, right? To imagine that _any_ half serious search engine isn't special-casing any referrer information it can get it's hands on from the top half of http://www.alexa.com/topsites http://www.alexa.com/topsites is surely naive?

nikster 16y ago

Well, no. Imagine I am the Bing toolbar and I am spying on my users. The Google search result for fake word zxczxczxczxczx is then a web page with a single link, and the user I am observing is clicking that link. So I am Bing, and I make a connection: zxczxczxczxczx (which appears on this web page) is related to said link. Since the word is artificial and doesn't appear anywhere else, it's eventually going to produce that page as a search result. So I think the author is right - what Google has done is they have proven that Bing toolbar tracks what users are doing, and sends the results back to Bing to improve their search. They have not proven that Bing treats google.com differently from any other page. For that they should have seeded 100 random web pages from different domains with artificial words, and have these pages contain a link which the user with Bing toolbar installed then clicks. I bet that it would yield the same result as the Google test, e.g. those fake words would get matched to the linked web pages.

barrkel 16y ago
zxczxczxczxczx isn't on the web page. It's on the search results page, but that page isn't for crawling, by robots.txt. Either MS is ignoring robots.txt, or they're parsing the URLs.
- bigiain 16y ago
  From my reading of the article, zxczxczxczxczx isn't in the search results page either, it was special cased at google to display the honeypot page for that specific query. But is Bing "parsing the url" any different to what Google's doing when I go into Google Analytics -> Traffic Sources -> Search Engines, and select Keyword in the second dropdown (after source)? Google are clearly showing me the search terms parsed out of the referrer urls from people who found my site in Bing. I think this is perhaps not obvious behaviour to the general public, but surely pretty much anyone who goes to the trouble of installing the Bing (or Google) toolbar has worked out for themselves that they're choosing to send data like this to Microsoft (and Google), and that it'll get used to "improve search" if it's found to be useful for that?
  - Terretta 16y ago

user24 16y ago

> parsing the query out of the url Or parsing any words found in the URL. Not unreasonable to believe.

moultano 16y ago
If that were the explanation, I would think they would come out and say it. (Maybe they will shortly.) Even so, I find it hard to believe that they wouldn't notice that most of the benefit of the technique comes from recreating google results, at which point we're back at the original ethical question of whether that's ok.
- dunham 16y ago
  most of the benefit of the technique comes from recreating google results I'm not 100% sure this is the case. Consider that an user wants a chicken enchilada recipe. 1. They go to a site that they trust for high quality data. (I like food&wine, saveur, epicurious, rick bayless for mexican, ...) 2. They search for "chicken enchilada" 3. They select the most appealing result (not necessarily the first one), based on author, snippet, rating, photo, etc. Domain knowledge. If you can associate (via the referrer) "chicken enchilada" with that page, you've encapsulated a lot of information - manual selection of site, manual selection of a page within that site. It's potentially useful (especially for long tail - people who have to go directly to sites because search engines fail them). Is there too much noise? Perhaps - you'd really have to look at the raw data to find out. Maybe you only want to look at "q=", "search=", and whatever phpBB, etc use. Maybe the signal that rises out of that noise is valuable. One big downside is that the results would be biased towards the preferences of people who install toolbars. :) The other downside is that SEOs could game the system by feeding MS bad data. (True if they're only looking at google data, too.)
forgotusername 16y ago
Not only does it not seem unreasonable, but for a company like Google, I find it incredibly hard to believe that at some point they wouldn't have tried it themselves, at least as an experiment.

othermaciej 16y ago

I think their terms of use say they may pass along the contents of form fields. On a Google search results page, your search query is sitting in a text field.

moultano 16y ago
Suppose it uses all text fields on the web without treating google specially. Being as charitable as possible here, I'm willing to believe that some well-meaning engineer coded this up without special casing google. Tested it out, found that it worked amazingly well, and then launched it. This seems unlikely, but possible. However, somewhere along the line someone must have known that the biggest benefit of this signal was recreating Google results. I'm not willing to believe that no one figured this out even if it wasn't the initial intent. At which point there's an ethical dilemma. At Google, a system like this wouldn't launch. Regardless of the mechanism, I don't believe that nobody at bing knows that this is what was going on. Maybe it's a cynical attempt to get around robots.txt. Maybe it's an honest mistake that gradually became a dishonest mistake, but I'm not willing to believe that they are oblivious.
- greendestiny 16y ago
  There is a huge difference between recreating Google results and incorporating a Google honeypot link into their results through legitimate means. One link isn't a search result, a search result is an ordered list of results. If its whole search results then Bing's got some questions to answer and deserves some bad press. Otherwise, not so much.
- othermaciej 16y ago
  Really? I doubt replicating Google long tail queries is the biggest benefit. User clicks seem like a useful signal of relevance for the same reason links from other sites are a useful signal of relevance.
  - moultano 16y ago
    Long tail queries are the ones which Google could easily demonstrate the effect without any confounding variables. I'd imagine this is affecting all of their ranking, as much as their own click data weighted by volume.

ddemchuk 16y ago

Considering they're a search engine, whose job is to crawl the internet and determine the main keywords of any random page on the internet, I would imagine they have text parsing abilities that make it possible to infer the topic of any arbitrary page.

toolate 16y ago

IE (with certain settings on) is sending page data back to Microsoft. If it sends the URL, title and referrer back then the following session is pretty easy to reverse engineer. 1. URL: "test - Google" Title: http://www.google.com/search?q=test 2. URL: http://test.org.us Title: "Test" Referrer: http://www.google.com/search?q=test Time: 2 secs 2. URL: http://en.wikipedia.org/wiki/Test_cricket Title: "Test cricket - Wikipedia, the free encyclopedia" Referrer: http://www.google.com/search?q=test Time: 249 secs It's really just an extension of page rank by seeing what links are being clicked on and not just which links exist. Whether MS should be capturing this data under false pretenses is another issue.

eddieplan9 16y ago
If this is the case, then it's rather easy to stop Microsoft from doing this. Just use POST instead of GET in the search page if you detect the browser is IE8. The referrer will always be the generic http://www.google.com/search http://www.google.com/search with no search term information.
- rjvir 16y ago
  Using POST instead of GET is not a good idea for a search results page. Most likely, the user would have to click through a dialog box ("are you sure you want to resubmit this form") every time the browser back button is used to return to the search results. Even if Google used redirects to circumvent this problem, searches won't be saved in the browser's history, which is a bit inconvenient. Changing from POST to GET in IE8 would stop Microsoft from mining data in the short term, but would drastically decrease the UX of Google for a large portion of its users who could very easily switch to Bing.
  - eftpotrm 16y ago
    OK, so for IE8 users return a single page AJAX app instead as a variant of what's already done with Instant. Still no referrers but no POST warning messages (which, BTW, have to be one of the most annoying things about ASP.Net web forms - postback was a boneheaded design decision from someone who didn't understand HTTP.)

ac2u 16y ago

"They're either parsing the query out of the url, or violating robots.txt to fetch the result page, almost certainly the former." Where's your evidence for any of that? There's another way. Store the input of any Text Box which is immediately followed by a form submit,(search term), and then store the href of the subsequent click (the result which the user finds helpful). The above is a way of handling it (naive and abstract) which doesn't target Google, and in fact would work for insite searches as well, or any search engine. Since you've already made up your mind that your two ways were the only way of doing things, what's your opinion given the above?

sonoffett 16y ago
I'm not sure why you are being voted down, the situation could be this. Also, Bing toolbar could simply be collecting tuples "(search_string_entered_in_toolbar, subsequent_href_clicked)" regardless of the search engine the user has specified and phoning these home... this seems like the easiest implementation (no parsing nonsense) and does not target google in particular.

bambax 16y ago

> If it were just click data, how would they get the terms? Exactly. It's not "click data" at all. It is monitoring user behavior on search engines, using both the clicks and the queries. Maybe it's not monitoring just Google as a search engine (although we have no proof of that yet: it seems it's just watching Google) but given Google's market share in search it doesn't make much difference. From the article: > I don’t even work in search and I could spot the real situation This sentence is at the same time arrogant and funny. He doesn't work in search, but he's certain he's spotted "the real situation". How did he fact check it, besides asking himself if he was correct and answering "yes, obviously I'm right. I'm always right -- and I don't even know anything! I amaze myself."

user24 16y ago
> certain he's spotted "the real situation". How did he fact check it By reading the official MS blog post[1], watching the Farsight conference live, and being confident in my own judgements based on the observed evidence. No other conclusion is supported by the evidence - you say "It is monitoring user behavior on search engines" - where's the evidence that it's exclusively search engines they're monitoring? Let alone exclusively Google? There has been no evidence yet presented. Therefore all we can conclude from Google's sting is that Bing use URL/click data. Which is exactly what I said in the original thread[2], in my blog post, and lo and behold it's what MS later said. That's how I know it's the real situation. [1] like where Harry says "A small piece of that is clickstream data we get from some of our customers, who opt-in ... To be clear, we learn from all of our customers". [2] http://news.ycombinator.com/item?id=2165947 http://news.ycombinator.com/item?id=2165947
- Terretta 16y ago
  You checked it against the assertion of the most interested party? It's not "click" data, it's the correlation between search term and SERP. That someone visited such and such a page after a click isn't the issue. That someone Googled for a unique term before clicking is at issue. Grandparent: http://news.ycombinator.com/item?id=2168377 http://news.ycombinator.com/item?id=2168377 Googling with Bing: http://www.collegehumor.com/video:1915736 http://www.collegehumor.com/video:1915736