6 ms·

Why is it that hard for engineers to rely on good old attribution? If every Google's featured snippet started its reply with "Breitbart says..." or "Trent Onli

Why is it that hard for engineers to rely on good old attribution? If every Google's featured snippet started its reply with "Breitbart says..." or "Trent Online, the leading Internet Newspaper in Nigeria said...", it wouldn't matter so much for those inevitable cases when the reply is taken straight from a white-supremacist or radical anarchist forum. The problem comes when the same reply is provided as "Google's true answer to the question" without further caveats.

askmike 10y ago
Because prepending that doesn't solve the problem Google wants to solve: once Google figures out that you typed in a question, they want to give you the answer to your question. Not tell you what other people think the answer is. (I agree that it is broken in a lot of cases now, but prepending that is a very bad workaround).
- conjectures 10y ago
  However prepending is possible. Figuring out 'the true answer' is in many cases impossible (what is the meaning of life, is x beautiful etc) and in many others merely several orders of magnitude harder than current tech can cope with (is there a polynomial time solution to TSP, why is public discourse so vacuous these days). So if that's what Google is gunning for it's going to be broken for a long time.
- TuringTest 10y ago
  > once Google figures out that you typed in a question, they want to give you the answer to your question And that's the problem that the article rightfully denounces. Having the answer to every question is not something that you can automate, not without strong AI. > Prepending that is a very bad workaround I strongly disagree; for me it's a very sensible thing to do - and in fact is how we do it in a social context ("I'm totally not making this shit up, I've read it in a scientific article / this morning newspaper / a tweet message"). We rely on the context where the knowledge came from as a proxy for how accurate its content might be. The default posture of engineers to a problem of inaccuracy is "make it more accurate". That's the logical method when you're building bridges, where we can describe the physics to an extremely accurate degree and where it would fall out or quickly degrade if you make a mistake. But for problems that rely on inaccurate or uncertain information, and where no exact models exists, the only reasonable way to build them is to degrade gracefully. If you don't handle failures, it will fail in spectacular ways - because it will fail sometimes. The system should provide an escape hatch, and allow the user to be in control. The user of an automated system should always be able to inspect how the system came to the provided result, and be able to override the given results in those cases where it will get it wrong (at least by some users in a supervising role, which other users could notify to). Unless a revolution happens, Artificial Intelligence will only ever work in the large if we put humans in the decision loop.
  - amarant 10y ago
    They do always append a link below their answer to the site where they got it. Personally I find this to be the exact same solution as yours (not literally). The problem is some people don't clicks links anymore. It's a problem people are stupid, but its not googles problem, even if idiots uses google (like everyone else).
    - TuringTest 10y ago
      > Personally I find this to be the exact same solution as yours (not literally). It may be the same for you (if you remember to follow the link for every single result that you get from Google, ever), but it's not enough for a mass-targeted product; the way in which the information is laid out and its relative prominence is essential to the way it will be perceived by the majority of the people using the service. This is not people being stupid, is how people's brain is hardwired to be efficient and only care about information that looks important from the way it's presented - rather than having to make a rational analysis of the relative importance of every information snippet that you find in your way. An upfront warning "this fact is brought to you by..." will have much more impact than a small, semi-hidden link that you have to remember to press, even if both convey the same information. (Moreover, the link is not available in the voice interface.)
  - askmike 10y ago
    > not without strong AI. You might be right if Google wants to answer all questions ever, but they can just limit this feature to situations where they do know what the answer is (you already see this when you type in math, conversions between different units, etc). > and in fact is how we do it in a social context ("I'm totally not making this shit up, I've read it in a scientific article / this morning newspaper / a tweet message"). That's not how I converse with people around me at all, I only talk about the source if there is some disbelieve about whether it is true. ---- When someone asks me for some information, I will answer and that is the context the person places me in. I won't tell that person "I heard this from Craig, he heard if from Judy and she was told by Bob, etc..". Google wants to become this person / context. The whole idea is that you don't have to click / verify / think about the potential webpage, because Google has already ranked all webpages that mention your query, and it will only snippet the top one (which is the result Google thinks will most likely answer your question). Adding the source is: - extra noise (the user now needs to look at the text & look at who said it and determine whether it might be true or not). - easily spoofable: you either rely on some meta tag for an Author, which websites can spoof. Or on a URL, which will lead to people doing stuff like "nytimes.com.quotefromgoogle.com". > The system should provide an escape hatch, and allow the user to be in control. Below the snippet are the 10 links that you can still click, functionality available since the mid 90s. > The user of an automated system should always be able to inspect how the system came to the provided result Just click the top link?
- return0 10y ago
  > Not tell you what other people think the answer is. Actually google and pagerank started like that.
interpol_p 10y ago
Prepending the source won't stop people from giving weight to the answer. Because it's the first and only one. You would probably risk giving more credibility to sites like Breitbart by crediting them with the top result, as people will come to associate those site names with answers given "by Google." If Google gives you a top search result, it means Google trusts that result to be a good one. There's no getting around the fact that people trust Google to give them good information, and Google's algorithms are failing to provide trustworthy information.
- DougWebb 10y ago
  If Google gives you a top search result, it means Google trusts that result to be a good one. Google doesn't "trust". That kind of thinking is the problem. Google has computed a ranking of the search results based on a bunch of factors, none of which involve any kind of cognitive analysis or judgement. The rest of your statement is absolutely right; people trust Google's ranking to imply an authority that it simply can't provide, because it's magic to them.
  - deong 10y ago
    Google isn't just an algorithm. It's a corporate entity with user-facing software, and while the algorithm has no notion of "trust", I would argue the company is demonstrating trust in any source that it presents as an answer in one of these bubbles. We're all agreeing that they shouldn't in many cases, but I do think they're doing so today. I don't care what your internal reasoning is. If I ask you a question and you present your answer as a fact, you're either trusting your information is correct or you're willfully lying to me. I think Google's defense here would be that they intend these answers to be interpreted by an intelligent adult. If it tells me that mermaids are real, I'll have a laugh about it and move on with my day. If it tells huge groups of apparently functioning adults that President Obama is instituting Sharia law, and those people believe it unquestioningly, well, I'm not sure Google is equipped to operate in that world.
  - TuringTest 10y ago
    > people trust Google's ranking to imply an authority that it simply can't provide, because it's magic to them. More the reason to expose the relevant parts of the "magic" as much as possible, rather than working as a black box and presenting the result as appearing out from nowhere.
BearGoesChirp 10y ago
Why does the source where the reply is taken from matters? I only see a few ways this works. The answer is correct. In this case, the source doesn't matter. The answer is incorrect. In this case, the source doesn't matter, there is a flaw in the entire concept. The answer's correctness depends upon the source. In this case, we have a deeper problem with the entire meaning of what a fact is. Personally, I actually go with option three, but I think the deeper discussion needs to happen before we begin to even consider if something like featured snippets is worth doing. The best way for me to sum up on feelings on this is that while facts are absolute, any piece of information is either true or false (at least within the realm of normal human interactions), it is impossible to separate any fact from a human's perception of the fact which will automatically introduce biases. An example: A few weeks ago I read online that <CNN anchor> did <very bad crime>. (I'll leave out the details because it only detracts from the point.) This is a factual statement, I did read it online, as did many others. But if many people who read it online states they read about it online, it creates a perception of truth to it, when the original post and the original replies were all showing how easy it is to create a false character assassination story. Regardless of how factual the statement is (it is true that I did read it online), my stating of such would still be an attempt to mislead even though I was using 100% factual statements. I do not see any automatic truth determiner being possible until our society realizes the biased nature of human known facts.
- TuringTest 10y ago
  > I only see a few ways this works. > The answer is correct. In this case, the source doesn't matter. > The answer is incorrect. In this case, the source doesn't matter, there is a flaw in the entire concept. > The answer's correctness depends upon the source. In this case, we have a deeper problem with the entire meaning of what a fact is. Again, why is it so hard for engineers to grasp the concept and importance of graceful degradation? A service doesn't need to be perfect to be useful, but if it's not perfect, it shouldn't burn your house when it fails. > The best way for me to sum up on feelings on this is that while facts are absolute, any piece of information is either true or false (at least within the realm of normal human interactions) People from Humanities would like to have a talk with you.
findateamfirst 10y ago
The attribution comes at the end
- Ajedi32 10y ago
  And when you ask Google Assistant, it audibly credits the source up-front, just like the parent comment was suggesting: "According to <site here>... <snippet text>"
- tyingq 10y ago
  It uses the site title versus a URL/domain. Which tends to favor shady sources. And, it's often below the fold.
pjc50 10y ago
http://www.weeklystandard.com/trumps-wiretap-claims-what-we-know-and-what-we-dont/article/2007096 http://www.weeklystandard.com/trumps-wiretap-claims-what-we-... "White House sources acknowledge that Trump had no idea whether the claims he was making were true when he made them." Here we have great attribution and a high-profile source (Trump, the source of claims about being wiretapped by Obama). It's just that the source doesn't care whether the claims are true or not.
jasonthevillain 10y ago
Well, for example, "According to The Vaccine Research Institution..." Now I just made up that organization. But how can an algorithm tell if it's a legitimate medical institute that develops vaccines or anti-vaxxer propaganda? If the algorithm actually had a way to distinguish the two, it could blacklist bullshit just as easily as it could disclose it.
- Karunamon 10y ago
  The problem there is that, at the end of the day, it's a fallible human putting stuff in the "bullshit blacklist". It also means that you're evaluating a statement based on its source rather than its merits, which is.. bad. When you're the #1 source of information on stuff in the world, that's an awful lot of power that can be trivially abused (e.g. "let's just blacklist unfavorable coverage of this lawsuit against us").
  - TuringTest 10y ago
    > you're evaluating a statement based on its source rather than its merits, which is.. bad. If you don't know the source, you have no way of knowing its merits, as you won't be aware of how the people came up with that knowledge.
    - Karunamon 10y ago
      By all means, show the source - just don't blacklist things based on the source. Truth is truth regardless of who speaks it.
tannhaeuser 10y ago
Fully agree that Google could present cues related to from whom a snippet was received (for example, by putting the snippet in a speech bubble), along with "what others say" complementary/discourse links. In fact, they should change their entire search result layout so as to add an "epistemological" (ie. related to how the presented result was obtained) knowledge dimension in the visual presentation, and enrich the search result with eg. links to employees, parent companies, and other known affiliations. That would be a strong move and help fight fake reviews, astroturfing, and fake news campaigning. And it would be much easier to implement compared to strong AI, if the latter is possible at all. In any case, it would be much more useful than the "searched 3.123e8 pages in 0.3 ms" non-information we get now. Not sure whether posts by verified/accredited people (real names) should be ranked higher in search results. Traditionally, hackers have preferred pseudonyms, and there are very good reasons to stay anonymous in oppressive regimes. OTOH, using pseudonyms/fake accounts at scale got us into the kind of political and other mass manipulation we have now.
tyingq 10y ago
I suspect they tested and tweaked what sort of format kept the most number of clicks and page views on Google owned sites. Prominent attribution wouldn't help in that regard. These kg entries are great for Google. They present the most likely sought out info, so little reason to click through. As a bonus, the vertical space they use pushes down any organic results that might be better answers.

askmike 10y ago

Because prepending that doesn't solve the problem Google wants to solve: once Google figures out that you typed in a question, they want to give you the answer to your question. Not tell you what other people think the answer is. (I agree that it is broken in a lot of cases now, but prepending that is a very bad workaround).

conjectures 10y ago
However prepending is possible. Figuring out 'the true answer' is in many cases impossible (what is the meaning of life, is x beautiful etc) and in many others merely several orders of magnitude harder than current tech can cope with (is there a polynomial time solution to TSP, why is public discourse so vacuous these days). So if that's what Google is gunning for it's going to be broken for a long time.
TuringTest 10y ago
> once Google figures out that you typed in a question, they want to give you the answer to your question And that's the problem that the article rightfully denounces. Having the answer to every question is not something that you can automate, not without strong AI. > Prepending that is a very bad workaround I strongly disagree; for me it's a very sensible thing to do - and in fact is how we do it in a social context ("I'm totally not making this shit up, I've read it in a scientific article / this morning newspaper / a tweet message"). We rely on the context where the knowledge came from as a proxy for how accurate its content might be. The default posture of engineers to a problem of inaccuracy is "make it more accurate". That's the logical method when you're building bridges, where we can describe the physics to an extremely accurate degree and where it would fall out or quickly degrade if you make a mistake. But for problems that rely on inaccurate or uncertain information, and where no exact models exists, the only reasonable way to build them is to degrade gracefully. If you don't handle failures, it will fail in spectacular ways - because it will fail sometimes. The system should provide an escape hatch, and allow the user to be in control. The user of an automated system should always be able to inspect how the system came to the provided result, and be able to override the given results in those cases where it will get it wrong (at least by some users in a supervising role, which other users could notify to). Unless a revolution happens, Artificial Intelligence will only ever work in the large if we put humans in the decision loop.
- amarant 10y ago
  They do always append a link below their answer to the site where they got it. Personally I find this to be the exact same solution as yours (not literally). The problem is some people don't clicks links anymore. It's a problem people are stupid, but its not googles problem, even if idiots uses google (like everyone else).
  - TuringTest 10y ago
    > Personally I find this to be the exact same solution as yours (not literally). It may be the same for you (if you remember to follow the link for every single result that you get from Google, ever), but it's not enough for a mass-targeted product; the way in which the information is laid out and its relative prominence is essential to the way it will be perceived by the majority of the people using the service. This is not people being stupid, is how people's brain is hardwired to be efficient and only care about information that looks important from the way it's presented - rather than having to make a rational analysis of the relative importance of every information snippet that you find in your way. An upfront warning "this fact is brought to you by..." will have much more impact than a small, semi-hidden link that you have to remember to press, even if both convey the same information. (Moreover, the link is not available in the voice interface.)
- askmike 10y ago
  > not without strong AI. You might be right if Google wants to answer all questions ever, but they can just limit this feature to situations where they do know what the answer is (you already see this when you type in math, conversions between different units, etc). > and in fact is how we do it in a social context ("I'm totally not making this shit up, I've read it in a scientific article / this morning newspaper / a tweet message"). That's not how I converse with people around me at all, I only talk about the source if there is some disbelieve about whether it is true. ---- When someone asks me for some information, I will answer and that is the context the person places me in. I won't tell that person "I heard this from Craig, he heard if from Judy and she was told by Bob, etc..". Google wants to become this person / context. The whole idea is that you don't have to click / verify / think about the potential webpage, because Google has already ranked all webpages that mention your query, and it will only snippet the top one (which is the result Google thinks will most likely answer your question). Adding the source is: - extra noise (the user now needs to look at the text & look at who said it and determine whether it might be true or not). - easily spoofable: you either rely on some meta tag for an Author, which websites can spoof. Or on a URL, which will lead to people doing stuff like "nytimes.com.quotefromgoogle.com". > The system should provide an escape hatch, and allow the user to be in control. Below the snippet are the 10 links that you can still click, functionality available since the mid 90s. > The user of an automated system should always be able to inspect how the system came to the provided result Just click the top link?
return0 10y ago
> Not tell you what other people think the answer is. Actually google and pagerank started like that.

interpol_p 10y ago

Prepending the source won't stop people from giving weight to the answer. Because it's the first and only one. You would probably risk giving more credibility to sites like Breitbart by crediting them with the top result, as people will come to associate those site names with answers given "by Google." If Google gives you a top search result, it means Google trusts that result to be a good one. There's no getting around the fact that people trust Google to give them good information, and Google's algorithms are failing to provide trustworthy information.

DougWebb 10y ago
If Google gives you a top search result, it means Google trusts that result to be a good one. Google doesn't "trust". That kind of thinking is the problem. Google has computed a ranking of the search results based on a bunch of factors, none of which involve any kind of cognitive analysis or judgement. The rest of your statement is absolutely right; people trust Google's ranking to imply an authority that it simply can't provide, because it's magic to them.
- deong 10y ago
  Google isn't just an algorithm. It's a corporate entity with user-facing software, and while the algorithm has no notion of "trust", I would argue the company is demonstrating trust in any source that it presents as an answer in one of these bubbles. We're all agreeing that they shouldn't in many cases, but I do think they're doing so today. I don't care what your internal reasoning is. If I ask you a question and you present your answer as a fact, you're either trusting your information is correct or you're willfully lying to me. I think Google's defense here would be that they intend these answers to be interpreted by an intelligent adult. If it tells me that mermaids are real, I'll have a laugh about it and move on with my day. If it tells huge groups of apparently functioning adults that President Obama is instituting Sharia law, and those people believe it unquestioningly, well, I'm not sure Google is equipped to operate in that world.
- TuringTest 10y ago
  > people trust Google's ranking to imply an authority that it simply can't provide, because it's magic to them. More the reason to expose the relevant parts of the "magic" as much as possible, rather than working as a black box and presenting the result as appearing out from nowhere.

BearGoesChirp 10y ago

Why does the source where the reply is taken from matters? I only see a few ways this works. The answer is correct. In this case, the source doesn't matter. The answer is incorrect. In this case, the source doesn't matter, there is a flaw in the entire concept. The answer's correctness depends upon the source. In this case, we have a deeper problem with the entire meaning of what a fact is. Personally, I actually go with option three, but I think the deeper discussion needs to happen before we begin to even consider if something like featured snippets is worth doing. The best way for me to sum up on feelings on this is that while facts are absolute, any piece of information is either true or false (at least within the realm of normal human interactions), it is impossible to separate any fact from a human's perception of the fact which will automatically introduce biases. An example: A few weeks ago I read online that <CNN anchor> did <very bad crime>. (I'll leave out the details because it only detracts from the point.) This is a factual statement, I did read it online, as did many others. But if many people who read it online states they read about it online, it creates a perception of truth to it, when the original post and the original replies were all showing how easy it is to create a false character assassination story. Regardless of how factual the statement is (it is true that I did read it online), my stating of such would still be an attempt to mislead even though I was using 100% factual statements. I do not see any automatic truth determiner being possible until our society realizes the biased nature of human known facts.

TuringTest 10y ago
> I only see a few ways this works. > The answer is correct. In this case, the source doesn't matter. > The answer is incorrect. In this case, the source doesn't matter, there is a flaw in the entire concept. > The answer's correctness depends upon the source. In this case, we have a deeper problem with the entire meaning of what a fact is. Again, why is it so hard for engineers to grasp the concept and importance of graceful degradation? A service doesn't need to be perfect to be useful, but if it's not perfect, it shouldn't burn your house when it fails. > The best way for me to sum up on feelings on this is that while facts are absolute, any piece of information is either true or false (at least within the realm of normal human interactions) People from Humanities would like to have a talk with you.

findateamfirst 10y ago

The attribution comes at the end

Ajedi32 10y ago
And when you ask Google Assistant, it audibly credits the source up-front, just like the parent comment was suggesting: "According to <site here>... <snippet text>"
tyingq 10y ago
It uses the site title versus a URL/domain. Which tends to favor shady sources. And, it's often below the fold.

pjc50 10y ago

http://www.weeklystandard.com/trumps-wiretap-claims-what-we-know-and-what-we-dont/article/2007096 http://www.weeklystandard.com/trumps-wiretap-claims-what-we-... "White House sources acknowledge that Trump had no idea whether the claims he was making were true when he made them." Here we have great attribution and a high-profile source (Trump, the source of claims about being wiretapped by Obama). It's just that the source doesn't care whether the claims are true or not.

jasonthevillain 10y ago

Well, for example, "According to The Vaccine Research Institution..." Now I just made up that organization. But how can an algorithm tell if it's a legitimate medical institute that develops vaccines or anti-vaxxer propaganda? If the algorithm actually had a way to distinguish the two, it could blacklist bullshit just as easily as it could disclose it.

Karunamon 10y ago
The problem there is that, at the end of the day, it's a fallible human putting stuff in the "bullshit blacklist". It also means that you're evaluating a statement based on its source rather than its merits, which is.. bad. When you're the #1 source of information on stuff in the world, that's an awful lot of power that can be trivially abused (e.g. "let's just blacklist unfavorable coverage of this lawsuit against us").
- TuringTest 10y ago
  > you're evaluating a statement based on its source rather than its merits, which is.. bad. If you don't know the source, you have no way of knowing its merits, as you won't be aware of how the people came up with that knowledge.
  - Karunamon 10y ago
    By all means, show the source - just don't blacklist things based on the source. Truth is truth regardless of who speaks it.

tannhaeuser 10y ago

Fully agree that Google could present cues related to from whom a snippet was received (for example, by putting the snippet in a speech bubble), along with "what others say" complementary/discourse links. In fact, they should change their entire search result layout so as to add an "epistemological" (ie. related to how the presented result was obtained) knowledge dimension in the visual presentation, and enrich the search result with eg. links to employees, parent companies, and other known affiliations. That would be a strong move and help fight fake reviews, astroturfing, and fake news campaigning. And it would be much easier to implement compared to strong AI, if the latter is possible at all. In any case, it would be much more useful than the "searched 3.123e8 pages in 0.3 ms" non-information we get now. Not sure whether posts by verified/accredited people (real names) should be ranked higher in search results. Traditionally, hackers have preferred pseudonyms, and there are very good reasons to stay anonymous in oppressive regimes. OTOH, using pseudonyms/fake accounts at scale got us into the kind of political and other mass manipulation we have now.

tyingq 10y ago

I suspect they tested and tweaked what sort of format kept the most number of clicks and page views on Google owned sites. Prominent attribution wouldn't help in that regard. These kg entries are great for Google. They present the most likely sought out info, so little reason to click through. As a bonus, the vertical space they use pushes down any organic results that might be better answers.