6 ms·

"... so Google wasn't able to cheat with privileged information about me as a user." The way they "cheated" was to look at the tens, hundreds, thousands, or te

by 3xblah 7y ago

"... so Google wasn't able to cheat with privileged information about me as a user." The way they "cheated" was to look at the tens, hundreds, thousands, or tens of thousands of previous queries that matched yours in substance. If the competing websites for "web search" had the same volume of submitted queries that are substantially similar to yours, then they too would be able to give you "exactly" what you are looking for "9 times out of 10". If all people submitting queries on the web are somehow convinced to visit only one website and submit a majority, (hypothetically let's say 93%) of their queries there, then it should be no surprise when that website "magically" starts becoming more "predictive" than other websites in returning "exactly" what searchers are looking for and becomes "by far the best at search".[1] The question raised by the blog post is whether enabling such "magic" is worth the trade-off of also creating a "panopticon" in that single website. Giving this level of visitation and query traffic to a single website makes any competing website (working off less than 7% of people's queries) seem irrelevant, maybe even pathetic. Without the enormous traffic, I would surmise the glory of the "empire" (to use the author's chosen term), and its appeal to "followers" (e.g., those who marvel at its "magic"), might dissipate rather quickly. 1. Especially the part of "search" that involves dealing with repeated, similar queries for popular information.

mantap 7y ago
Knowing what the most common questions are doesn't help you find the corresponding answers. At best you can find the result that users click on most frequently, but if all of the results in the result page suck then that's not going to help. Google's advantage is simply billions of dollars and 20 years of R&D into NLP tech.
- pizza234 7y ago
  > Knowing what the most common questions are doesn't help you find the corresponding answers. At best you can find the result that users click on most frequently I doubt that users have a strict expectation of finding the correct (interpretation of "corresponding") answer. Even if the most clicked answer[s] are still not correct, they may still be the ones who suck less, which can still be acceptable/desirable. Keep in mind that in this perspective, the concept is similar to Google Translate - Google built a translator that, at least originally, doesn't understand language, instead, it applies (applied) a static model on a large amount of documents. While they certainly poured a large amount of money on it, it's success can't be centered purely on the economical factor.
  - mantap 7y ago
    Well in the given example the user asks a question "who was the actor that..." and Google gives them the answer. This is really hard to achieve by accident no matter how much traffic you get. It requires NLP machinery to answer a question, it's not a matching problem but an understanding problem. There are at least three kinds of queries that a search engine has to handle. Requests for websites e.g. "Facebook". Traditional keyword searches across the web e.g. "Twitter ban political ads", and questions "Who was the guy who voiced bender?".
- Thorentis 7y ago
  > Knowing what the most common questions are doesn't help you find the corresponding answers. Not directly, but it does mean that, say, the second 50% of people that ask the same question will get a better answer than the first 50%. Google has been able to build fairly accurate instant results based on which sites users were clicking on before. I'd say that a majority of simple general knowledge queries are solved by quoting the first 3 sentences of the wikipedia page that match the search query. But let's say that there is no easy match to show a quick result for. But after 1000 queries, 98% of users clicked on one particular site on the first page and never went back to the search results. Google then A/B trials how many clicks result from showing that website at the top of the results page in an instant result window. If clicks drastically drop, that's a sign most users are satisfied with that result. They were only able to do that, because of the 1000s of times people typed that query and interacted with the site. So I'd say that yes, having common questions asked over and over again does help you find the answer that users are looking for. The funny / scary part here, is that this may not be the correct answer. But it's the answer that satisfies the most users, and is therefore the one that will keep the most people coming back to the Google search engine.
bko 7y ago
Just because google has seen my search before, how does that translate to being able to give me good search results? Search is an incredibly hard and nuanced problem. I doubt ddg's main disadvantage is training set of search queries. Plus, you're underestimating the percentage of queries that google has never seen before. Google processes trillions of searches every year, and still, 15% of those queries have never been seen by Google before. [0] [0] https://searchengineland.com/google-reaffirms-15-searches-new-never-searched-273786 https://searchengineland.com/google-reaffirms-15-searches-ne...
- 3xblah 7y ago
  "Plus, you're underestimating the percentage of queries that google has never seen before." No. I am highlighting that 75% are queries that they have seen before. Note also that the 15% is a figure that is declining, based on what it was in 2007.
nojvek 7y ago
There’s some truth to that but Google also has a massive knowledge graph. For imdb and freebase like queries they can give very accurate answers as cards. Other humans don’t need to teach every single fact. They just need to teach how to extract and organize facts from a site. When you see google giving you a one/two word answer as a card, that’s very likely coming from it’s knowledge graph. What we need is more of this open knowledge graphs. Google and Wolfram Alpha are both closed sources but have deep understanding in niche domains.
- 3xblah 7y ago
  I am not a Google search user so I am not sure what they are doing with "cards", however it sounds as though they are trying more to themselves become a "secondary source" (versus, e.g., just providing some "context" around the matched terms). The goal being that the user never has to leave the Google website. For example, say the film credits for Spiderman is a "primary source", and a cast list for Spiderman derived from the credits at IMDb is a "secondary source". Google extracts the information from IMDb and substitutes itself as the secondary source. Does this raise an issue in that ideally users should sometimes be retrieving information from (i.e., accessing) primary and secondary sources directly, whereas a third party always acting as a universal secondary source, e.g., a third party funded by advertising, might introduce (more) bias into the information retrieval process, e.g., in competition for "eyeballs". From the OP references: https://sparktoro.com/blog/less-than-half-of-google-searches-now-result-in-a-click/ https://sparktoro.com/blog/less-than-half-of-google-searches...
  - kinkrtyavimoodh 7y ago
    You might find this philosophically objectionable but the truth is that most people go to Google to find answers, not "primary or secondary sources", and are therefore better served by the answer fished out and presented to them. It's also another matter that so much of the internet is now filled with copy-pasted crap and artificially inflated content that it's actually hard to find what you are looking for on the so-called primary source webpages. If I want to find George Clooney's age, I don't want to sift through a 2000 page ad-ridden Page3-esque gossip blog about him. At the same time, when I want detailed info about something, I will go in and try to read through the primary material.

by 3xblah 7y ago

mantap 7y ago

Knowing what the most common questions are doesn't help you find the corresponding answers. At best you can find the result that users click on most frequently, but if all of the results in the result page suck then that's not going to help. Google's advantage is simply billions of dollars and 20 years of R&D into NLP tech.

pizza234 7y ago
> Knowing what the most common questions are doesn't help you find the corresponding answers. At best you can find the result that users click on most frequently I doubt that users have a strict expectation of finding the correct (interpretation of "corresponding") answer. Even if the most clicked answer[s] are still not correct, they may still be the ones who suck less, which can still be acceptable/desirable. Keep in mind that in this perspective, the concept is similar to Google Translate - Google built a translator that, at least originally, doesn't understand language, instead, it applies (applied) a static model on a large amount of documents. While they certainly poured a large amount of money on it, it's success can't be centered purely on the economical factor.
- mantap 7y ago
  Well in the given example the user asks a question "who was the actor that..." and Google gives them the answer. This is really hard to achieve by accident no matter how much traffic you get. It requires NLP machinery to answer a question, it's not a matching problem but an understanding problem. There are at least three kinds of queries that a search engine has to handle. Requests for websites e.g. "Facebook". Traditional keyword searches across the web e.g. "Twitter ban political ads", and questions "Who was the guy who voiced bender?".
Thorentis 7y ago
> Knowing what the most common questions are doesn't help you find the corresponding answers. Not directly, but it does mean that, say, the second 50% of people that ask the same question will get a better answer than the first 50%. Google has been able to build fairly accurate instant results based on which sites users were clicking on before. I'd say that a majority of simple general knowledge queries are solved by quoting the first 3 sentences of the wikipedia page that match the search query. But let's say that there is no easy match to show a quick result for. But after 1000 queries, 98% of users clicked on one particular site on the first page and never went back to the search results. Google then A/B trials how many clicks result from showing that website at the top of the results page in an instant result window. If clicks drastically drop, that's a sign most users are satisfied with that result. They were only able to do that, because of the 1000s of times people typed that query and interacted with the site. So I'd say that yes, having common questions asked over and over again does help you find the answer that users are looking for. The funny / scary part here, is that this may not be the correct answer. But it's the answer that satisfies the most users, and is therefore the one that will keep the most people coming back to the Google search engine.

bko 7y ago

Just because google has seen my search before, how does that translate to being able to give me good search results? Search is an incredibly hard and nuanced problem. I doubt ddg's main disadvantage is training set of search queries. Plus, you're underestimating the percentage of queries that google has never seen before. Google processes trillions of searches every year, and still, 15% of those queries have never been seen by Google before. [0] [0] https://searchengineland.com/google-reaffirms-15-searches-new-never-searched-273786 https://searchengineland.com/google-reaffirms-15-searches-ne...

3xblah 7y ago
"Plus, you're underestimating the percentage of queries that google has never seen before." No. I am highlighting that 75% are queries that they have seen before. Note also that the 15% is a figure that is declining, based on what it was in 2007.

nojvek 7y ago

There’s some truth to that but Google also has a massive knowledge graph. For imdb and freebase like queries they can give very accurate answers as cards. Other humans don’t need to teach every single fact. They just need to teach how to extract and organize facts from a site. When you see google giving you a one/two word answer as a card, that’s very likely coming from it’s knowledge graph. What we need is more of this open knowledge graphs. Google and Wolfram Alpha are both closed sources but have deep understanding in niche domains.

3xblah 7y ago
I am not a Google search user so I am not sure what they are doing with "cards", however it sounds as though they are trying more to themselves become a "secondary source" (versus, e.g., just providing some "context" around the matched terms). The goal being that the user never has to leave the Google website. For example, say the film credits for Spiderman is a "primary source", and a cast list for Spiderman derived from the credits at IMDb is a "secondary source". Google extracts the information from IMDb and substitutes itself as the secondary source. Does this raise an issue in that ideally users should sometimes be retrieving information from (i.e., accessing) primary and secondary sources directly, whereas a third party always acting as a universal secondary source, e.g., a third party funded by advertising, might introduce (more) bias into the information retrieval process, e.g., in competition for "eyeballs". From the OP references: https://sparktoro.com/blog/less-than-half-of-google-searches-now-result-in-a-click/ https://sparktoro.com/blog/less-than-half-of-google-searches...
- kinkrtyavimoodh 7y ago
  You might find this philosophically objectionable but the truth is that most people go to Google to find answers, not "primary or secondary sources", and are therefore better served by the answer fished out and presented to them. It's also another matter that so much of the internet is now filled with copy-pasted crap and artificially inflated content that it's actually hard to find what you are looking for on the so-called primary source webpages. If I want to find George Clooney's age, I don't want to sift through a 2000 page ad-ridden Page3-esque gossip blog about him. At the same time, when I want detailed info about something, I will go in and try to read through the primary material.