4 ms·

I find it interesting that Google couldn't get the Bill Murray one. No matter what combination of "did bill murray run for president" I search for, I get a rich

I find it interesting that Google couldn't get the Bill Murray one. No matter what combination of "did bill murray run for president" I search for, I get a rich snippet on the results page from snopes.com which says "Claim: Comedian Bill Murray is running for president and proclaimed religion to be "the worst enemy of mankind." Claimed by: Internet Fact check by Snopes.com: FALSE" It would seem they are reasonably close but this is more of a product integration failure than a recognition failure.

dahart 9y ago
Great point! I get the same from Google. Looking back in the article, he only criticized Siri and Cortana on this question. He claimed none of them got it right, but didn't say specifically what Google did with it, and it's entirely possible it was a different answer before now. Bigger picture though, Bill Murray is famous making it easier to answer questions like this. In general, does the wording of the author's question truly imply he's searching for a fact check, and do you expect Google to know that even if there are no articles that match the wording of the question? The snopes articles does contain the terms "did", and "Bill Murray", and "run for president", so we don't have any evidence that Google understands the question, we just have some content that matches the query. The issue I see is that the computational question of search has long been trying to measure relevance by matching the query against the corpus. This Bill Murray question is an example of how that can break down. I might actually want the fake articles... and I might not. There's no way for the search engine to know without making an inference, and the expectation that mass market search engines make inferences seems pretty new to me - and I don't expect that when I do text searching. I guess I just expect voice search to push the need for question understanding and inference making even faster than text search has.
- ancalimon 9y ago
  Heya, My first Hacker News inclusion. I feel like there should be some rite of passage. Well, other than the sudden and unanticipated login attempts. My testing device for Google was the Google Home speaker, which appears to have a different tolerance for reading search results. I've had it rattle off several sentences from web pages for other keywords in the list (see, for example, the boiling point of water), but for the Bill Murray question there seems to be some kind of limiter. I just re-checked, using the exact phrasing I had before, and it still says that it doesn't know, but it's learning all the time. I'm guessing there is some kind of a relevance check for the speaker version compared to the phone version. The phone is probably happier to return any result (a la Siri), whereas the speaker appears to be making some attempt to understand what I'm asking for before reading search results. This particular question appears to trigger the speaker not to read the search results. We can only speculate as to why: does it not find it relevant enough? Is there a reserved path on "Did xyz" questions when sent to Google Home? Am I unknowingly in the A/B testing group that doesn't get the answer? There's few ways of knowing black-box without massive data testing, but it is curious.
  - dahart 9y ago
    Welcome! I don't know of any rite of passage, but maybe I gave you your first upvote? ;) > I'm guessing there is some kind of a relevance check for the speaker version compared to the phone version. I would bet on that & expect it too... I'm sure all these voice search products are experimenting with how voice search needs to be tuned differently than text search.
- ewanm89 9y ago
  What does "Google Home" speaker usually do when there is no clear result? On the phone google assistant just displays google search results in such a case, obviously that can't work on the "Google Home" speaker.
  - ancalimon 9y ago
    Sometimes it'll read a page, e.g. it read a passage from Wikipedia a few times. But if it really can't decide, it'll say something like "I'm sorry, I don't know that one".
deleted 9y ago
[deleted]

dahart 9y ago

Great point! I get the same from Google. Looking back in the article, he only criticized Siri and Cortana on this question. He claimed none of them got it right, but didn't say specifically what Google did with it, and it's entirely possible it was a different answer before now. Bigger picture though, Bill Murray is famous making it easier to answer questions like this. In general, does the wording of the author's question truly imply he's searching for a fact check, and do you expect Google to know that even if there are no articles that match the wording of the question? The snopes articles does contain the terms "did", and "Bill Murray", and "run for president", so we don't have any evidence that Google understands the question, we just have some content that matches the query. The issue I see is that the computational question of search has long been trying to measure relevance by matching the query against the corpus. This Bill Murray question is an example of how that can break down. I might actually want the fake articles... and I might not. There's no way for the search engine to know without making an inference, and the expectation that mass market search engines make inferences seems pretty new to me - and I don't expect that when I do text searching. I guess I just expect voice search to push the need for question understanding and inference making even faster than text search has.

ancalimon 9y ago
Heya, My first Hacker News inclusion. I feel like there should be some rite of passage. Well, other than the sudden and unanticipated login attempts. My testing device for Google was the Google Home speaker, which appears to have a different tolerance for reading search results. I've had it rattle off several sentences from web pages for other keywords in the list (see, for example, the boiling point of water), but for the Bill Murray question there seems to be some kind of limiter. I just re-checked, using the exact phrasing I had before, and it still says that it doesn't know, but it's learning all the time. I'm guessing there is some kind of a relevance check for the speaker version compared to the phone version. The phone is probably happier to return any result (a la Siri), whereas the speaker appears to be making some attempt to understand what I'm asking for before reading search results. This particular question appears to trigger the speaker not to read the search results. We can only speculate as to why: does it not find it relevant enough? Is there a reserved path on "Did xyz" questions when sent to Google Home? Am I unknowingly in the A/B testing group that doesn't get the answer? There's few ways of knowing black-box without massive data testing, but it is curious.
- dahart 9y ago
  Welcome! I don't know of any rite of passage, but maybe I gave you your first upvote? ;) > I'm guessing there is some kind of a relevance check for the speaker version compared to the phone version. I would bet on that & expect it too... I'm sure all these voice search products are experimenting with how voice search needs to be tuned differently than text search.
ewanm89 9y ago
What does "Google Home" speaker usually do when there is no clear result? On the phone google assistant just displays google search results in such a case, obviously that can't work on the "Google Home" speaker.
- ancalimon 9y ago
  Sometimes it'll read a page, e.g. it read a passage from Wikipedia a few times. But if it really can't decide, it'll say something like "I'm sorry, I don't know that one".

deleted 9y ago

[deleted]