4 ms·

Microsoft’s Bing uses Google search results–and denies it (2011)

56 points by ProfDreamer 8y ago

randyrand 8y ago
Bing and Google get their search results by scraping websites that other people own. For bing, google is just one of the website they scrape. So probably not illegal - though it definitely looks bad.
- engrefoobiz 8y ago
  There's a difference in a search engine scraping a site just to link to that site. And scraping a site to claim its content as your own.
aogl 8y ago
This is really bizarre that MS(Bing) can still claim that they don't use Google results with such an investigation and fake results showing identically in both places, having originated on the Google side. Come on Bing, just own up!
- dmix 8y ago
  Bing's excuse (at the time in 2011) was that it was the result of Internet Explorer, Bing toolbar, and other user behaviour within their products which seeded this data. I'm curious if the Google engineers visited any of the 100 search terms using IE or Microsoft service... but I doubt that. There had to be some level of scraping. It's pretty embarrassing. They also were using autocomplete terms that matched Google's perfectly, which is what initially caught Google's attention.
  - notamy 8y ago
    > I'm curious if the Google engineers visited any of the 100 search terms using IE or Microsoft service... but I doubt that. From the article: >We gave 20 of our engineers laptops with a fresh install of Microsoft Windows running Internet Explorer 8 with Bing Toolbar installed. As part of the install process, we opted in to the “Suggested Sites” feature of IE8, and we accepted the default options for the Bing Toolbar. > We asked these engineers to enter the synthetic queries into the search box on the Google home page, and click on the results, i.e., the results we inserted. We were surprised that within a couple weeks of starting this experiment, our inserted results started appearing in Bing. Below is an example: a search for [hiybbprqag] on Bing returned a page about seating at a theater in Los Angeles. As far as we know, the only connection between the query and result is Google’s result page (shown above).
justinsaccount 8y ago
So they.. * Inserted results into the search index for odd queries like "hiybbprqag" * Searched Bing for those queries * Saw the result from google Wouldn't it have been easier and just as definitive to just * Search Bing for hiybbprqag * Show that google received a search query for the same thing
- belltaco 8y ago
  No, that was never how it worked, or Google would have sued and probably won. What actually happened was that, if a user had Bing's search bar installed in IE, the fine print for the toolbar installation declared that the keywords and links that they click on from other search engines could be sent to Microsoft's servers. From the article: >We gave 20 of our engineers laptops with a fresh install of Microsoft Windows running Internet Explorer 8 with Bing Toolbar installed. As part of the install process, we opted in to the “Suggested Sites” feature of IE8, and we accepted the default options for the Bing Toolbar. So, the toolbar would send keyword, url, ie. {"alsdkjfsd","example.com/alsdkjfsd.html"} to Bing's crawler. Bing's crawler would include that in their search index after crawling it. Perhaps the rank of the page in the search results was included as well. This was a way for Bing to discover unindexed pages. One could argue that that it's scummy but legal, and that the user also technically owns the data of ("what I searched for", "what URL I visited"), rather than Google being the exclusive owner of that data, and could send that to third parties. That could be a reason Google did not take legal action over it. Google's IE toolbar also sent a ton of data to Google. https://searchengineland.com/turning-the-tables-on-the-google-toolbar-disclosure-claims-63596 https://searchengineland.com/turning-the-tables-on-the-googl...
  - justinsaccount 8y ago
    Ah, the toolbar! Now it makes more sense.
- dunpeal 8y ago
  With the massive size of their infrastructure, I'm guessing it would be very difficult to conclusively identify a single query, let alone attribute it to Bing. Also, the process you describe isn't conclusive, since it's possible that someone else randomly searched for "hiybbprqag". Finally, it's a stronger argument the way it is. Otherwise, it's just Google's word against MS. With their method, they could show that they could predict the search results by sealing a prediction before running the search on Bing. It's independently verifiable.
deleted 8y ago
[deleted]
dmix 8y ago
Original thread on HN from 8yrs ago (511 upvotes): https://news.ycombinator.com/item?id=2167875 https://news.ycombinator.com/item?id=2167875
Someguywhatever 8y ago
This is 7 years old though, I would be curious to see if the experiment is still repeatable today. I've noticed personally that bing has gotten a lot better, but that could just be because it's still a delayed version of Google. Microsoft may even be using IE to harvest data to reverse engineer a Google search algorithm. Even if they make an inferior verision thats 90% as good, that's still good enough to keep them relevant in the market.
- exikyut 8y ago
  Well, you could sort of do this experiment yourself, with a bit of preparation and a decent bit of patience. - Prepare a new small website on an extremely obscure topic; something a search engine will consider uninteresting and not worth ranking highly, but just legitimate enough to warrant maintaining in the search index. On top of obscurity, don't link to the site from anywhere. This is so the site doesn't wind up crawled by Bing. - Manually add the site to Google's index (I forget where but there's a webmaster tool that can be used to manually add things; then you wait and see if the crawler paid any attention and added it to the index) - Add a single webpage on the site that, in amongst all the other words, contains a single random word, or a series of random words. Now repeatedly search Google for the term, on a machine nowhere near a Windows system, and and wait see if your site shows up. It will take some time to get this working, because unlike Google's fully contrived test, actually getting garbage/nonsense words into the index does take effort, as (in my experience) the index seems to be primed to index/prefer valid words over garbage; on top of this, your nonsense word will be (if you're doing it right) the only result in the world for the word, so it may be the search result is in the index but has some attribute marking it below a "show in results" threshold, if such metrics exist. - Now you have your magic term, do what Google did, and repeatedly search Bing for the term, from IE. - ??? - Profit?
  - belltaco 8y ago
    That experiment would have always failed. Check my other comment https://news.ycombinator.com/item?id=17810832 https://news.ycombinator.com/item?id=17810832
nickthemagicman 8y ago
This is so typical Microsoft. Zune, Xbox, Bing, literally 90% of what they do is a rip off of other innovators ideas.
matt4077 8y ago
I actually think Bing is ahead of Google for image search these days.
partiallypro 8y ago
If people are having to dig up 7 year old articles to bash Microsoft, I guess they've improved a lot.
- beaner 8y ago
  Probably related to the other thread on HN today about a guy having his site scraped by a competitor.
Grue3 8y ago
IIRC they literally installed a Bing toolbar and agreed to turn on a feature that indexes all the sites that the user visits. To then blame Bing for indexing their decoy page, when Google themselves gave permission to index it, is absolutely laughable.
O1111OOO 8y ago
In essence, sounds like a bit of keylogging, data-sharing with external site(s). From the article, even Google couldn't exactly figure what was going on technically. Makes me wonder what else Microsoft was recording and sharing at the time - or was it just limited to google.com.

56 points by ProfDreamer 8y ago

Bing and Google get their search results by scraping websites that other people own. For bing, google is just one of the website they scrape. So probably not illegal - though it definitely looks bad.

engrefoobiz 8y ago
There's a difference in a search engine scraping a site just to link to that site. And scraping a site to claim its content as your own.

aogl 8y ago

This is really bizarre that MS(Bing) can still claim that they don't use Google results with such an investigation and fake results showing identically in both places, having originated on the Google side. Come on Bing, just own up!

dmix 8y ago
Bing's excuse (at the time in 2011) was that it was the result of Internet Explorer, Bing toolbar, and other user behaviour within their products which seeded this data. I'm curious if the Google engineers visited any of the 100 search terms using IE or Microsoft service... but I doubt that. There had to be some level of scraping. It's pretty embarrassing. They also were using autocomplete terms that matched Google's perfectly, which is what initially caught Google's attention.
- notamy 8y ago
  > I'm curious if the Google engineers visited any of the 100 search terms using IE or Microsoft service... but I doubt that. From the article: >We gave 20 of our engineers laptops with a fresh install of Microsoft Windows running Internet Explorer 8 with Bing Toolbar installed. As part of the install process, we opted in to the “Suggested Sites” feature of IE8, and we accepted the default options for the Bing Toolbar. > We asked these engineers to enter the synthetic queries into the search box on the Google home page, and click on the results, i.e., the results we inserted. We were surprised that within a couple weeks of starting this experiment, our inserted results started appearing in Bing. Below is an example: a search for [hiybbprqag] on Bing returned a page about seating at a theater in Los Angeles. As far as we know, the only connection between the query and result is Google’s result page (shown above).

justinsaccount 8y ago

So they.. * Inserted results into the search index for odd queries like "hiybbprqag" * Searched Bing for those queries * Saw the result from google Wouldn't it have been easier and just as definitive to just * Search Bing for hiybbprqag * Show that google received a search query for the same thing

belltaco 8y ago
No, that was never how it worked, or Google would have sued and probably won. What actually happened was that, if a user had Bing's search bar installed in IE, the fine print for the toolbar installation declared that the keywords and links that they click on from other search engines could be sent to Microsoft's servers. From the article: >We gave 20 of our engineers laptops with a fresh install of Microsoft Windows running Internet Explorer 8 with Bing Toolbar installed. As part of the install process, we opted in to the “Suggested Sites” feature of IE8, and we accepted the default options for the Bing Toolbar. So, the toolbar would send keyword, url, ie. {"alsdkjfsd","example.com/alsdkjfsd.html"} to Bing's crawler. Bing's crawler would include that in their search index after crawling it. Perhaps the rank of the page in the search results was included as well. This was a way for Bing to discover unindexed pages. One could argue that that it's scummy but legal, and that the user also technically owns the data of ("what I searched for", "what URL I visited"), rather than Google being the exclusive owner of that data, and could send that to third parties. That could be a reason Google did not take legal action over it. Google's IE toolbar also sent a ton of data to Google. https://searchengineland.com/turning-the-tables-on-the-google-toolbar-disclosure-claims-63596 https://searchengineland.com/turning-the-tables-on-the-googl...
- justinsaccount 8y ago
  Ah, the toolbar! Now it makes more sense.
dunpeal 8y ago
With the massive size of their infrastructure, I'm guessing it would be very difficult to conclusively identify a single query, let alone attribute it to Bing. Also, the process you describe isn't conclusive, since it's possible that someone else randomly searched for "hiybbprqag". Finally, it's a stronger argument the way it is. Otherwise, it's just Google's word against MS. With their method, they could show that they could predict the search results by sealing a prediction before running the search on Bing. It's independently verifiable.

deleted 8y ago

[deleted]

dmix 8y ago

Original thread on HN from 8yrs ago (511 upvotes): https://news.ycombinator.com/item?id=2167875 https://news.ycombinator.com/item?id=2167875

Someguywhatever 8y ago

This is 7 years old though, I would be curious to see if the experiment is still repeatable today. I've noticed personally that bing has gotten a lot better, but that could just be because it's still a delayed version of Google. Microsoft may even be using IE to harvest data to reverse engineer a Google search algorithm. Even if they make an inferior verision thats 90% as good, that's still good enough to keep them relevant in the market.

exikyut 8y ago
Well, you could sort of do this experiment yourself, with a bit of preparation and a decent bit of patience. - Prepare a new small website on an extremely obscure topic; something a search engine will consider uninteresting and not worth ranking highly, but just legitimate enough to warrant maintaining in the search index. On top of obscurity, don't link to the site from anywhere. This is so the site doesn't wind up crawled by Bing. - Manually add the site to Google's index (I forget where but there's a webmaster tool that can be used to manually add things; then you wait and see if the crawler paid any attention and added it to the index) - Add a single webpage on the site that, in amongst all the other words, contains a single random word, or a series of random words. Now repeatedly search Google for the term, on a machine nowhere near a Windows system, and and wait see if your site shows up. It will take some time to get this working, because unlike Google's fully contrived test, actually getting garbage/nonsense words into the index does take effort, as (in my experience) the index seems to be primed to index/prefer valid words over garbage; on top of this, your nonsense word will be (if you're doing it right) the only result in the world for the word, so it may be the search result is in the index but has some attribute marking it below a "show in results" threshold, if such metrics exist. - Now you have your magic term, do what Google did, and repeatedly search Bing for the term, from IE. - ??? - Profit?
- belltaco 8y ago
  That experiment would have always failed. Check my other comment https://news.ycombinator.com/item?id=17810832 https://news.ycombinator.com/item?id=17810832

nickthemagicman 8y ago

This is so typical Microsoft. Zune, Xbox, Bing, literally 90% of what they do is a rip off of other innovators ideas.

matt4077 8y ago

I actually think Bing is ahead of Google for image search these days.

partiallypro 8y ago

If people are having to dig up 7 year old articles to bash Microsoft, I guess they've improved a lot.

beaner 8y ago
Probably related to the other thread on HN today about a guy having his site scraped by a competitor.

Grue3 8y ago

IIRC they literally installed a Bing toolbar and agreed to turn on a feature that indexes all the sites that the user visits. To then blame Bing for indexing their decoy page, when Google themselves gave permission to index it, is absolutely laughable.

O1111OOO 8y ago

In essence, sounds like a bit of keylogging, data-sharing with external site(s). From the article, even Google couldn't exactly figure what was going on technically. Makes me wonder what else Microsoft was recording and sharing at the time - or was it just limited to google.com.