9 ms·
I had a front row seat for this test. I believe the experiment we ran provides conclusive proof. I'm on a panel with a representative from Bing later today and
by Matt_Cutts 16y ago
I had a front row seat for this test. I believe the experiment we ran provides conclusive proof. I'm on a panel with a representative from Bing later today and I'll ask Bing about this directly.
- kgermino 16y agoPlease keep us posted as best you can.
- Matt_Cutts 16y agoI believe the search panel with Google and Bing will be live-streaming, so anyone can tune in. It's by a group called Big Think.
- illdave 16y agoClickable: http://bigthink.com/series/62 http://bigthink.com/series/62
- cool-RR 16y agoI couldn't find the streaming video from there. Did anyone find it?
- Splines 16y agoAccording to the description it'll be live at 10AM PST (a little before this comment is 2 hours old).
- illdave 16y agoI guess they'll put the streaming video up when it starts (10am PST, so in just under 2 hours). That seemed like the most appropriate page to link to.
- deleted 16y ago[deleted]
- lda 16y agoLooks like the site can't handle the traffic as of now (1.20pm ET).
- deleted 16y ago[deleted]
- raganwald 16y agoThe article suggests that the Bing toolbar monitors what its user click and uses that information to improve Bing search results. Is that what you have conclusively proved? I'm interested in another experiment. If you set up a honeypot, search for the term, but never click on the link, does the honeypot start showing up in Bing? The article doesn't say whether you tried this. Did you try it? Are Bing scraping your results from the page or only tracking their users clicks?
- Matt_Cutts 16y agoAnyone can test that Microsoft's software sends the clicks back to Microsoft, although I believe Microsoft sends the data back by SSL, so it's harder to verify even that than you'd expect. Google's search results are blocked in robots.txt, so I don't believe Bing has been able to crawl our search results directly. All the evidence points to users' clicks on Google, which are then sent to Microsoft. Microsoft has (so far) declined to admit whether our allegation is true. Getting them to talk about exactly what they do and what software they use or don't use would be the easiest way. I'd like them to confirm or deny, which is why I wanted to go to this search panel later today and ask them.
- poacher69 16y agoare you claiming that google never scrapes bing search results pages? or any other search result pages?
- illdave 16y agoFrom my experience, Googlebot doesn't crawl pages that are blocked in robots.txt files. Check out Bing's robots.txt: http://bing.com/robots.txt http://bing.com/robots.txt - notice how /search is disallowed. That typically means that Googlebot isn't able to access that page. The same for the other search engines, it's more down to if they specify (through robots.txt) that Googlebot isn't allowed to crawl those results.
- Matt_Cutts 16y agopoacher69, we crawl the public web. Anyone that blocks us out with robots.txt, we won't crawl. If you check bing.com/robots.txt, it has "Disallow: /search" . So no, we won't crawl Bing's search results pages. If anything, users tend to complain when search results from Lycos or wherever show up in Google.
- justlearning 16y agoI can't argue about the legal vs illegal. I sure would like to tell the bing team that they should better google in their attempts. This strategy is mere copying into- 'look, I am as good as google'. Microsoft never learns from past do they??
- brudgers 16y agoGiven that Google appears to have an active program to monitor the results of search queries on Bing and to track Bing's page rankings and the ways in which they change over time in (How else is this more likely to have come to their attention?), they should hardly be shocked, shocked to find Microsoft doing something similar. [edit] I have always suspected that the real value of Bing for Microsoft is to prevent Google's data mining of queries originating in Redmond.
- user24 16y ago... and feeding that data back into their own search results, rather than just using it for analysis to see how your competitors are performing? If I'm in the business of giving horse racing tips and I read your tips to see what your strike rate is compared to mine, that's one thing. If I start tipping the same horses as you, purely because you tipped them, that's quite another thing.
- brudgers 16y agoMicrosoft isn't using strike rate data in the important sense which you imply - the strike rate for Google is advertising revenue. It's not as if Microsoft is collecting info on the advertisements displayed and then soliciting those advertisers to spend their dollars on Bing (at least that's not part of the allegations). I am pretty confident that Google feeds every bit of legally collected relevant data back into their search algorithms.
- user24 16y agono, the "strike rate" is precision/recall. Good advertising CTRs is a side-effect of relevancy. Relevancy is measured by precision and recall. edit to expand: If the measure by which a search engine evaluated itself was advertising revenues, they'd all have massive intrusive adverts, and no users. The only viable measure can be the quality of the search results themselves. As a happy coincidence, if you build something capable of delivering high quality results, you can very easily use that to produce highly relevant adverts. Imagine that each advert is like a little webpage, and rank them just the same as you do for normal webpages. (caveat: there's no link graph for adverts, so we're reduced to using a simpler text mining approach, eg bag of words vector space la-di-da).
- user24 16y agoI have a suspicion what you'll find is that Bing use the toolbar to match $current_page_content with $clicked_page_content. When $current_page_content contains obscure words, that becomes the only signal, and so bing's engine will naturally associate it with $clicked_page. In other words, there's a relationship between Page A and Page B if there exists a link beween them (==PageRank). But the strength of the relationship is increased based on how many users click on that link. I think that's the information Bing were trying to capture (or if they weren't, they should have been). What I'm saying is that it's probably an unintentional side-effect. At scale though, the effect is that Bing gradually uses Google as a signal, simply because Google is a popular site. edit: Yet another way of saying it: I think it's not just clicks on Google searches that are captured by Bing, but clicks anywhere. Google is a large site, so its influence on Bing can be measured. This is what we're seeing. My theory. I don't work in search.
- mvandemar 16y agoThis makes a lot of sense, and would have be easy enough for Google to test as well, creating some tiny, brand new, never before heard of test search engine that Bing would have no reason to copy, see if the same thing happened.
- user24 16y agoThat would be a nice way of testing it. edit: I'm not even sure if it's only search engines that are being analysed by Bing or all pages, but it's possible that it is just SEs - they could be capturing query terms distinctly.
- recoiledsnake 16y agoAlso, why did the experiment succeed for only 6 or 7 of the 100 terms that they tried? There's more than what meets the eye here, regardless of the hype and everyone jumping on the bandwagon.
- mvandemar 16y agoThey also probably should have tested to see if it happened with results other than the ones in the #1 spot.
- podperson 16y agoBing copying Google's search results is just evil. Not like copying Apple's iPhone design and user interface and giving it away to Apple's competitors, which is good. Right? It's ugly and immoral and probably legal. Good job catching Microsoft at it (and I think the really really unethical and scary bit is that Microsoft is cheerfully stealing info from users via their browser). I also realize that Google got where it is in Search by innovation and iteration, and that Google's search team has nothing to do with Android per se, but you might see how Apple people feel about the business empires built on stealing their ideas.
- nlogn 16y agoI don't think Android devices being similar in some ways to the iPhone is even remotely analogous to what Bing is doing here. One is called healthy competition (and I don't see how Android is copying iOS). The other is literally just copying data.
- othermaciej 16y agoThe original Android phone design looked like this: http://tctechcrunch.files.wordpress.com/androidphone.jpg http://tctechcrunch.files.wordpress.com/androidphone.jpg Then iPhone came out and it looked like this: http://km.support.apple.com/library/APPLE/APPLECARE_ALLGEOS/HT3939/HT3939--iphone_dimensions-001-en.png http://km.support.apple.com/library/APPLE/APPLECARE_ALLGEOS/... Now Android phones look like this: http://farm3.static.flickr.com/2795/4208849005_dd4b608729.jpg http://farm3.static.flickr.com/2795/4208849005_dd4b608729.jp... You don't see any signs of copying here?
- magicalist 16y agoturtles all the way down, my friend: http://1.bp.blogspot.com/_YgpHa1oeHME/TTxZAZWLaxI/AAAAAAAAC_g/mXieqgNA0YE/s1600/Palm+pilot.jpg http://1.bp.blogspot.com/_YgpHa1oeHME/TTxZAZWLaxI/AAAAAAAAC_...
- podperson 16y ago
- sjs382 16y agoIs there a chance this panel is being recorded? I'd love to be a fly on that wall :)
- sjs382 16y agoOops, sorry. Found the link elsewhere in this thread! http://news.ycombinator.com/item?id=2165861 http://news.ycombinator.com/item?id=2165861
- encoderer 16y agoIt's embarrassing for Google to complain about this. You FINALLY get a little competition on your turf and you try to make some big issue that, as a market leader, the product you produce is being watched, analyzed and in some ways incorporated by your competitor. There is no victim here. They are not taking your 1st result and copying it. They are taking the result the user clicked. Obviously you didn't predict that with your algorithm or you'd have always made that the 1st result. Instead, what they're tracking is user behavior, not your raw ranking. Obviously users give Google implicit permission to track their behavior by using your product. And similarly, by installing the Bing toolbar, they're giving Bing that permission. This is beneath you Matt and it's beneath Google.
- TimothyBurgess 16y agoAs we all know, this isn't the first time Microsoft has copied someone else. And I'm sure it won't be the last. I think Google has a right to complain. Microsoft has resorted to these less than innovative tactics to monopolize themselves for a long time now, and it isn't fair to companies like Google who have worked their butts off (and gave 1.8 million shares - $336M in 2005 - to Stanford for the PageRank algorithm) to develop their superior product.
- alttab 16y agoI disagree. From some point of view somewhere this is "standing on the shoulders of giants." Perspective changes things here, which means no one is "right" or "wrong".
- recoiledsnake 16y agoErr, by that line of thinking, Google leveraged Linux (the hard work of volunteers) to earn tens of billions and does not release the modified code for use of the volunteers. Of course they are not required to, but it isn't fair to Linux developers who have worked their butts off to develop Linux.
- xuki 16y agoI strongly disagree, it's not the same analogy. Linux developers explicit say you can use the code for free. It's more like Linus say you can't use the code, but Google use them anyway. Btw, Google contributed a lot to open source projects.
- Vlasta 16y agoI usually criticize Microsoft, but I am on their side in this case. Why shouldn't Microsoft be using this kind of data? Google search result pages are part of the internet just like any other publicly available web site. Microsoft monitors what the users are clicking on Google and probably on Bing and other sites. So what? Monitoring users is not a new thing. It may be unethical and I may personally hate it, but almost everyone is doing it. Google should stop whining about this and make their search result the best they can. If they had the best search engine, Bing could come close, but never overcome Google by just copying part of it.
- KirinDave 16y agoI can't help but feel a little surprised that Google found it ethical to lie on their search results page for any reason; and clearly this trojan horse page was a lie. Certainly at every search engine I've worked at we always said, "We can put up adds and help, but we can never outright lie." I suppose that's ameliorated by the fact that this was an internal experiment, but still.. The temptation to abuse that power is pretty big.
- bluelu 16y agoYou are just jealous that your engineers haven't thought of this yourself. I bet is just recording general searches (input query) + clicked links. A pretty good idea. And don't tell me you are not using the results from the google toolbar to rank the sites in google search.
- kenjackson 16y agoGreat panel session. One of the best I've seen. I think you made your point, although I think MS did a good job neutralizing it too. My question for you Matt... is there any way for Google to build a toolbar that effectively does what the Bing toolbar does (or even a joint one?). I jump to use the various search engines because no single search engine is sufficient. But clearly when Google isn't sufficient, you don't get the value of when I go to Bing. And vice-versa (as I don't use the Bing toolbar currently, but that may change now). Or do you feel that with 65% of the market, you don't need this info?