4 ms·

Google didn't get worse, the internet did and Google just reflects the junk we put in the pile.

psyc 4y ago
It would be naive to think 100,000 of the most elite practitioners at the company that became the verb for what it does working on the problem for 20 years would be able to keep up.
marginalia_nu 4y ago
I built my search engine in part to test this hypothesis, and think the result is that it's wrong. A large chunk of the internet, as it used to be, is still there. You just can't find it in the sea of aggressively search engine optimized (and often machine generated) junk that it has to compete with on Google. You can't find it on Reddit or Facebook or Twitter either, because none of their algorithms favor that sort of content. This creates an illusion that it doesn't exist. But it does.
- vosper 4y ago
  If those sites were trying to SEO your search engine, rather than Google, then they’d be the only results you saw, too.
  - marginalia_nu 4y ago
    Oh yeah, absolutely. I think a big part of the problem is Google's dominant position essentially puts them in an impossible uphill battle against an entire industry attempting to game their algorithms. The solution to this problem is search engine diversity. If you have multiple search engines with incompatible algorithms and disparate views on quality, then even if you manage to game one of the algorithms effectively, that search engine will just lose popularity and your SEO will be less useful. The winning strategy becomes to attempt to SEO against multiple contradictory algorithms, which is something that will never be as successful as manipulating Google alone.
    - mountainriver 4y ago
      Also just subscription search?
arinlen 4y ago
> Google didn't get worse, the internet did and Google just reflects the junk we put in the pile. Given Google's role in pushing stuff like Google's Accelerated Mobile Pages (AMP), I'd say that Google's role in degrading the world's internet experience goes way beyond being an innocent bystander.
GuB-42 4y ago
The "good" internet is still there, better than ever. Sure, there is a lot of crap, more than ever, but finding the good stuff in the pile of crap is what search engines like Google are for. I don't know why Google search results appear to be worse. Bing (and its many proxies) are not getting better either, so it is probably not only Google's fault. Among the explanation I can think of: - There is a cognitive bias: results are not actually worse - There is an arms race between search engines and people trying to game the results (SEO), and SEO is winning - There is a recent disconnect between what is in best interest for Google (and Microsoft) to show you and what is best for the user. - We are reaching some technical limits. We went from simple indexing, to PageRank, to deep learning, constantly improving, and now we are stagnating. - Google (and Microsoft) is suffering the large company syndrome, which is like SEO form the inside: employees optimize for their paycheck instead of the quality of the end product, and the corporate structure can't make these goals match. What I don't believe: - Google is turning evil: Google is not evil, Google just wants to make money, it has always been the case, and giving out great results made Google a trillion dollar company. AFAIK, Google is not owned by Satan, it has no reason to "turn evil" unless it makes them money, and why it would abandon a "not evil" strategy that worked so well in the past requires some explanation. - There is a simple solution: Google may be a corporate mess, but people there are not completely stupid. If there is a simple solution that fits in a random HN post, they would have done it. As they say, for every complex problem there is an answer that is clear, simple, and wrong.
- ColinHayhurst 4y ago
  We take the view that Google and Bing are becoming answer engines. Answers on the SERPs suppress discovery on, and navigation of the web. A real search engine has organic results that minimise your time on the SERPs, providing a diversity of relevant, useful destinations to click on with enough info (snippets & title) to decide where to explore. You also need an independent index and crawler (spider that follows links). Otherwise you are a proxy service or a correlation to a search engine. Whether you use a real search engine API or scraping or query-click ML model(s) you are not independent and really a search service. We've taken the hard road; our search quality is patchy (now with an index >5 billion), but we are one of few genuine search engines if you think that matters.
s1artibartfast 4y ago
Google broke exact search terms. It literally won't let you search the string you want. That was a choice
rurban 4y ago
nope. I switched to duckduckgo, and the amount of spam is much less. it's a bit harder to find the best articles, but it's still better than the average google search.

psyc 4y ago

It would be naive to think 100,000 of the most elite practitioners at the company that became the verb for what it does working on the problem for 20 years would be able to keep up.

marginalia_nu 4y ago

I built my search engine in part to test this hypothesis, and think the result is that it's wrong. A large chunk of the internet, as it used to be, is still there. You just can't find it in the sea of aggressively search engine optimized (and often machine generated) junk that it has to compete with on Google. You can't find it on Reddit or Facebook or Twitter either, because none of their algorithms favor that sort of content. This creates an illusion that it doesn't exist. But it does.

vosper 4y ago
If those sites were trying to SEO your search engine, rather than Google, then they’d be the only results you saw, too.
- marginalia_nu 4y ago
  Oh yeah, absolutely. I think a big part of the problem is Google's dominant position essentially puts them in an impossible uphill battle against an entire industry attempting to game their algorithms. The solution to this problem is search engine diversity. If you have multiple search engines with incompatible algorithms and disparate views on quality, then even if you manage to game one of the algorithms effectively, that search engine will just lose popularity and your SEO will be less useful. The winning strategy becomes to attempt to SEO against multiple contradictory algorithms, which is something that will never be as successful as manipulating Google alone.
  - mountainriver 4y ago
    Also just subscription search?

arinlen 4y ago

> Google didn't get worse, the internet did and Google just reflects the junk we put in the pile. Given Google's role in pushing stuff like Google's Accelerated Mobile Pages (AMP), I'd say that Google's role in degrading the world's internet experience goes way beyond being an innocent bystander.

GuB-42 4y ago

The "good" internet is still there, better than ever. Sure, there is a lot of crap, more than ever, but finding the good stuff in the pile of crap is what search engines like Google are for. I don't know why Google search results appear to be worse. Bing (and its many proxies) are not getting better either, so it is probably not only Google's fault. Among the explanation I can think of: - There is a cognitive bias: results are not actually worse - There is an arms race between search engines and people trying to game the results (SEO), and SEO is winning - There is a recent disconnect between what is in best interest for Google (and Microsoft) to show you and what is best for the user. - We are reaching some technical limits. We went from simple indexing, to PageRank, to deep learning, constantly improving, and now we are stagnating. - Google (and Microsoft) is suffering the large company syndrome, which is like SEO form the inside: employees optimize for their paycheck instead of the quality of the end product, and the corporate structure can't make these goals match. What I don't believe: - Google is turning evil: Google is not evil, Google just wants to make money, it has always been the case, and giving out great results made Google a trillion dollar company. AFAIK, Google is not owned by Satan, it has no reason to "turn evil" unless it makes them money, and why it would abandon a "not evil" strategy that worked so well in the past requires some explanation. - There is a simple solution: Google may be a corporate mess, but people there are not completely stupid. If there is a simple solution that fits in a random HN post, they would have done it. As they say, for every complex problem there is an answer that is clear, simple, and wrong.

ColinHayhurst 4y ago
We take the view that Google and Bing are becoming answer engines. Answers on the SERPs suppress discovery on, and navigation of the web. A real search engine has organic results that minimise your time on the SERPs, providing a diversity of relevant, useful destinations to click on with enough info (snippets & title) to decide where to explore. You also need an independent index and crawler (spider that follows links). Otherwise you are a proxy service or a correlation to a search engine. Whether you use a real search engine API or scraping or query-click ML model(s) you are not independent and really a search service. We've taken the hard road; our search quality is patchy (now with an index >5 billion), but we are one of few genuine search engines if you think that matters.

s1artibartfast 4y ago

Google broke exact search terms. It literally won't let you search the string you want. That was a choice

rurban 4y ago

nope. I switched to duckduckgo, and the amount of spam is much less. it's a bit harder to find the best articles, but it's still better than the average google search.