5 ms·

Study Suggests Google Harms Consumers by Skewing Search Results

29 points by caminante 11y ago

dpflan 11y ago
From yesterday: https://news.ycombinator.com/item?id=9798256 https://news.ycombinator.com/item?id=9798256 No comments there, maybe some will arise here.
- raldi 11y ago
  It was also posted here, with a few dozen comments: https://news.ycombinator.com/item?id=9796813 https://news.ycombinator.com/item?id=9796813
- moey 11y ago
  Why does it matter? I'm tired of seeing people link other discussions. If it's a live active discussion, thats one thing, but a dead discussion from yesterday?....
  - jemka 11y ago
    The comments of a submission are valuable to those other than the participants. To readers that don't or won't comment, the concept of 'live active discussion' is moot. In fact, I could argue that it's most beneficial for a reader to wait until the comment activities die down. Either way the link to another similar thread is an odd thing to be tired of.
- caminante 11y ago
  D'oh! I searched "Yelp":last-24-hours before posting and your submission was right outside the threshold. My bad!
  - dpflan 11y ago
    No worries, man. I think the comments are the most valuable part of posting. So actually my linking to a submission that yielded no comments was not valuable, but I guess it did lead another user to share a post that did have a discussion. Your submission had the right timing and sparked a discussion!
bitmapbrother 11y ago
I thought Tim Wu had integrity. Looks like a paycheck from Yelp was all it took.
- pasta_2 11y ago
  lol It couldn't possibly be that Google did anything wrong.
  - bitmapbrother 11y ago
    Yelp is desperate. Why not do a proper independent study instead of paying people to say what you want to hear?
- Zigurd 11y ago
  Probably not just Yelp. Rupert has issues with Google.
themeekforgotpw 11y ago
If you aren't worried about the customer case you need be aware of nation state interest in partnering with search engines and content curators - not just to censor - but to deceive and manipulate. The leaked US Special Forces document on Wikileaks does not mentioned Google but does talk about putting adversaries in an information bubble that will make it look like they've lost public support for their cause. They expand this bubble around not only the adversary, but their family and close friends. The way the US military works in large and increasing part is that it works with US corporations to achieve these effects. Given the complaints by other countries of weaponization of Google/Facebook and studies done on using search engine results to manipulate voting we should keep our eyes peeled that Google continues to 'not be evil'.
- secfirstmd 11y ago
  Interesting...Which one exactly? Source?
  - themeekforgotpw 11y ago
    If you are asking about vote manipulation: https://www.google.com/search?q=search+engine+manipulate+votes https://www.google.com/search?q=search+engine+manipulate+vot... https://www.google.com/search?q=Search+Engine+Manipulation+Effect https://www.google.com/search?q=Search+Engine+Manipulation+E... https://www.google.com/search?q=Lok+Sabha+election+research https://www.google.com/search?q=Lok+Sabha+election+research http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4004139/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4004139/
    - secfirstmd 11y ago
      Thanks. And the document about US SF units doing this?
      - themeekforgotpw 11y ago
        https://wikileaks.org/wiki/US_Special_Forces_counterinsurgency_manual_analysis https://wikileaks.org/wiki/US_Special_Forces_counterinsurgen...
hayksaakian 11y ago
This article is awfully vague on the details of the problem. On what basis do they decide whether results are harmful or helpful?
- jdp23 11y ago
  They used clickthrough rates as a proxy for how useful the result was.
- jccalhoun 11y ago
  yelp didn't show up as high in results as they think they should so they paid for a study that shockingly agreed with them...
- eridius 11y ago
  I think it's this: > The study’s authors—[…]—found that users were 45% more likely to click on results that were ranked purely by relevance, rather than as Google ranks them now, with its own services displayed prominently. If users are clicking on results 45% less often, that suggests they aren't getting results that are as useful to them. From that it seems fairly reasonable to claim that this is harmful to consumers.
deleted 11y ago
[deleted]
googtardsz 11y ago
Don't be (harmfully) evil?
downandout 11y ago
The key statement in this article, that renders this report largely useless, is of course contained at the very bottom of it: "The authors conceded that there were differences between their click surveys and actual search results, and that they didn’t have access to internal Google data to verify their results." In other words, their conclusions are meaningless because they simply don't have access to to the data to determine how Google would rank search results in the fictional scenario that they were attempting to study. They essentially did A/b testing, showing some users fictional, Yelp-heavy search results injected into a fake Google results page, and compared those CTR's to those of an actual Google results page. While the fake pages achieved a higher CTR, that would probably be the case for many custom-crafted results pages curated by humans. This was a "study" that was paid for by Yelp, in their quest to force Google to feature their results more prominently. I guess when you build your entire business on quasi-extrotion, heavy handed tactics come naturally. But this one falls completely flat for me.
- ocdtrekkie 11y ago
  Shouldn't it be a highly poignant here, that only Google has access to the data necessarily to police Google's conduct?
- eridius 11y ago
  Maybe I'm reading it differently than you, but I don't see that statement as being particularly meaningful. All that says is that they didn't have access to Google's actual internal data on clickthrough rates. Which is not surprising. It turns out what they did was use a site called UsabilityHub (which is intended for testing website designs) to show the study participants a screenshot of search results. One screenshot showed Google's results, while another shows search results that were purportedly sorted purely by relevance. I'm assuming users simply reported whether they would have clicked on any results on the page, but the article doesn't go into that level of detail. I also assume the users weren't told whether they were looking at the genuine results or not. Given that, there's nothing at all surprising about that statement you highlighted. In fact, unless Google were willing to actually run the study themselves, randomly showing either their current results or results sorted purely by "relevance", and gathering the real clickthrough rates, then what the article describes here seems like the only real reasonable way to do this.
eridius 11y ago
Can anyone speak as to how likely it is for this study to be unbiased? It was sponsored by Yelp, and Yelp obviously has an agenda to push. The fact that it was sponsored by them does not necessarily mean that the study was influenced, but it does raise that possibility.
- ocdtrekkie 11y ago
  It's reasonable to take it with a grain of salt. Particularly to give a very suspicious eye to their research methodologies.
sytelus 11y ago
This is basically study done by Yelp employees saying that Google results for local intent queries include more from Google+ than on Yelp and people participated in their study didn't liked that. Well... Yelp obviously wants to monetize their (ummm, sorry, users content) content. I would think company like Google wouldn't have much problem paying Yelp but probably their demands are too much (just my guess). So Google showed them the door and decided to do their own thing and now Yelp has problem with that. For companies like Yelp, I would suggest they respect users intent and rights. Millions of users have sweated on to create all these content on Yelp. They shouldn't be building a giant walled garden around it. Instead they should just make all data available with a fair license to anyone who wants to use it. When more companies like Apple, Samsung etc buys their licences, it might be overall better for them than to get one huge deal out of one company. May be they should even make their data available for free for research (they did made small part available, but I'm saying why not all of it?). When the community works on this data, lot of cool ideas can spark which again benefit them to get more licensing deals and would make it worth for users who sweated out to create all this content.
deleted 11y ago
[deleted]
acconrad 11y ago
Let us assume it does skew search results. Two things we also know are true: * Google is a publicly-owned corporation. They have every right, as they own the platform, to skew it as they see fit. They aren't supposed to be unbiased, if you wanted that you'd have to have a completely open-source search engine to make this argument credible. * Just because they own the lion's share of the search market does not mean you have to use them. Don't like Google? Use DuckDuckGo. Use Bing. Use AskJeeves! Are people using Google at gunpoint? Certainly not. This whole things feels weak - isn't it obvious that Google is owned by a corporation with it's own self-interest agenda, and by using that service, you are bound to the skew it may or may not put in it's searches / ads / everything?
- subpopular 11y ago
  Totally agree here. Google is not a public utility where everyone has the right to be represented equally or fairly.

29 points by caminante 11y ago

From yesterday: https://news.ycombinator.com/item?id=9798256 https://news.ycombinator.com/item?id=9798256 No comments there, maybe some will arise here.

raldi 11y ago
It was also posted here, with a few dozen comments: https://news.ycombinator.com/item?id=9796813 https://news.ycombinator.com/item?id=9796813
moey 11y ago
Why does it matter? I'm tired of seeing people link other discussions. If it's a live active discussion, thats one thing, but a dead discussion from yesterday?....
- jemka 11y ago
  The comments of a submission are valuable to those other than the participants. To readers that don't or won't comment, the concept of 'live active discussion' is moot. In fact, I could argue that it's most beneficial for a reader to wait until the comment activities die down. Either way the link to another similar thread is an odd thing to be tired of.
caminante 11y ago
D'oh! I searched "Yelp":last-24-hours before posting and your submission was right outside the threshold. My bad!
- dpflan 11y ago
  No worries, man. I think the comments are the most valuable part of posting. So actually my linking to a submission that yielded no comments was not valuable, but I guess it did lead another user to share a post that did have a discussion. Your submission had the right timing and sparked a discussion!

bitmapbrother 11y ago

I thought Tim Wu had integrity. Looks like a paycheck from Yelp was all it took.

pasta_2 11y ago
lol It couldn't possibly be that Google did anything wrong.
- bitmapbrother 11y ago
  Yelp is desperate. Why not do a proper independent study instead of paying people to say what you want to hear?
Zigurd 11y ago
Probably not just Yelp. Rupert has issues with Google.

themeekforgotpw 11y ago

If you aren't worried about the customer case you need be aware of nation state interest in partnering with search engines and content curators - not just to censor - but to deceive and manipulate. The leaked US Special Forces document on Wikileaks does not mentioned Google but does talk about putting adversaries in an information bubble that will make it look like they've lost public support for their cause. They expand this bubble around not only the adversary, but their family and close friends. The way the US military works in large and increasing part is that it works with US corporations to achieve these effects. Given the complaints by other countries of weaponization of Google/Facebook and studies done on using search engine results to manipulate voting we should keep our eyes peeled that Google continues to 'not be evil'.

secfirstmd 11y ago
Interesting...Which one exactly? Source?
- themeekforgotpw 11y ago
  If you are asking about vote manipulation: https://www.google.com/search?q=search+engine+manipulate+votes https://www.google.com/search?q=search+engine+manipulate+vot... https://www.google.com/search?q=Search+Engine+Manipulation+Effect https://www.google.com/search?q=Search+Engine+Manipulation+E... https://www.google.com/search?q=Lok+Sabha+election+research https://www.google.com/search?q=Lok+Sabha+election+research http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4004139/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4004139/
  - secfirstmd 11y ago
    Thanks. And the document about US SF units doing this?
    - themeekforgotpw 11y ago
      https://wikileaks.org/wiki/US_Special_Forces_counterinsurgency_manual_analysis https://wikileaks.org/wiki/US_Special_Forces_counterinsurgen...

hayksaakian 11y ago

This article is awfully vague on the details of the problem. On what basis do they decide whether results are harmful or helpful?

jdp23 11y ago
They used clickthrough rates as a proxy for how useful the result was.
jccalhoun 11y ago
yelp didn't show up as high in results as they think they should so they paid for a study that shockingly agreed with them...
eridius 11y ago
I think it's this: > The study’s authors—[…]—found that users were 45% more likely to click on results that were ranked purely by relevance, rather than as Google ranks them now, with its own services displayed prominently. If users are clicking on results 45% less often, that suggests they aren't getting results that are as useful to them. From that it seems fairly reasonable to claim that this is harmful to consumers.

deleted 11y ago

[deleted]

googtardsz 11y ago

Don't be (harmfully) evil?

downandout 11y ago

The key statement in this article, that renders this report largely useless, is of course contained at the very bottom of it: "The authors conceded that there were differences between their click surveys and actual search results, and that they didn’t have access to internal Google data to verify their results." In other words, their conclusions are meaningless because they simply don't have access to to the data to determine how Google would rank search results in the fictional scenario that they were attempting to study. They essentially did A/b testing, showing some users fictional, Yelp-heavy search results injected into a fake Google results page, and compared those CTR's to those of an actual Google results page. While the fake pages achieved a higher CTR, that would probably be the case for many custom-crafted results pages curated by humans. This was a "study" that was paid for by Yelp, in their quest to force Google to feature their results more prominently. I guess when you build your entire business on quasi-extrotion, heavy handed tactics come naturally. But this one falls completely flat for me.

ocdtrekkie 11y ago
Shouldn't it be a highly poignant here, that only Google has access to the data necessarily to police Google's conduct?
eridius 11y ago
Maybe I'm reading it differently than you, but I don't see that statement as being particularly meaningful. All that says is that they didn't have access to Google's actual internal data on clickthrough rates. Which is not surprising. It turns out what they did was use a site called UsabilityHub (which is intended for testing website designs) to show the study participants a screenshot of search results. One screenshot showed Google's results, while another shows search results that were purportedly sorted purely by relevance. I'm assuming users simply reported whether they would have clicked on any results on the page, but the article doesn't go into that level of detail. I also assume the users weren't told whether they were looking at the genuine results or not. Given that, there's nothing at all surprising about that statement you highlighted. In fact, unless Google were willing to actually run the study themselves, randomly showing either their current results or results sorted purely by "relevance", and gathering the real clickthrough rates, then what the article describes here seems like the only real reasonable way to do this.

eridius 11y ago

Can anyone speak as to how likely it is for this study to be unbiased? It was sponsored by Yelp, and Yelp obviously has an agenda to push. The fact that it was sponsored by them does not necessarily mean that the study was influenced, but it does raise that possibility.

ocdtrekkie 11y ago
It's reasonable to take it with a grain of salt. Particularly to give a very suspicious eye to their research methodologies.

sytelus 11y ago

This is basically study done by Yelp employees saying that Google results for local intent queries include more from Google+ than on Yelp and people participated in their study didn't liked that. Well... Yelp obviously wants to monetize their (ummm, sorry, users content) content. I would think company like Google wouldn't have much problem paying Yelp but probably their demands are too much (just my guess). So Google showed them the door and decided to do their own thing and now Yelp has problem with that. For companies like Yelp, I would suggest they respect users intent and rights. Millions of users have sweated on to create all these content on Yelp. They shouldn't be building a giant walled garden around it. Instead they should just make all data available with a fair license to anyone who wants to use it. When more companies like Apple, Samsung etc buys their licences, it might be overall better for them than to get one huge deal out of one company. May be they should even make their data available for free for research (they did made small part available, but I'm saying why not all of it?). When the community works on this data, lot of cool ideas can spark which again benefit them to get more licensing deals and would make it worth for users who sweated out to create all this content.

deleted 11y ago

[deleted]

acconrad 11y ago

Let us assume it does skew search results. Two things we also know are true: * Google is a publicly-owned corporation. They have every right, as they own the platform, to skew it as they see fit. They aren't supposed to be unbiased, if you wanted that you'd have to have a completely open-source search engine to make this argument credible. * Just because they own the lion's share of the search market does not mean you have to use them. Don't like Google? Use DuckDuckGo. Use Bing. Use AskJeeves! Are people using Google at gunpoint? Certainly not. This whole things feels weak - isn't it obvious that Google is owned by a corporation with it's own self-interest agenda, and by using that service, you are bound to the skew it may or may not put in it's searches / ads / everything?

subpopular 11y ago
Totally agree here. Google is not a public utility where everyone has the right to be represented equally or fairly.