4 ms·

> If I want my blog to show up on your search engine, do I have to get it linked by one of those sites, or can I register with you? Will I be tier 1, or I thin

> If I want my blog to show up on your search engine, do I have to get it linked by one of those sites, or can I register with you? Will I be tier 1, or I think what I'd say in defense is that we've misunderstood what search engines are useful for. They're really bad at helping us discover new things. Your blog might be awesome, but it's not going to be easy for a search engine to tell that it's awesome. It's going to have to compete with other blogs that also want views, some of whom are going to be better than yours at SEO, and so on. What a search engine might be able to tell is that it's useful. Because what search engines are at least potentially good at is answering questions. You do that by having a list of known good sites to answer specific types of questions, and looking at the sites they link to. It's when you try to do both (index everything on the web and provide accurate answers to specific questions) that you end up failing to do either. For example this is the #2 result for "python f strings" on DDG[1]. It's total garbage, and, quoting the blog, "we can do better". (This result is also on page 1 for the same query on Google.) What I believe ddevault is suggesting is that we make a search engine that does the only thing search engines are really good at, answering questions. You throw away the idea of indexing everything on the web, and therefore the possibility of "discovery". What that means is that in 2020 you need some other mechanism for discovering new sites, bloggers, and so on. Fortunately we do have some alternatives in that space. To be clear, I don't know if I 100% buy this argument, but I think it's the general idea behind what's being suggested in this blog post. [1] https://careerkarma.com/blog/python-f-string/ https://careerkarma.com/blog/python-f-string/

jrochkind1 6y ago
> You do that by having a list of known good sites to answer specific types of questions, and looking at the sites they link to. I mean, that's basically the core of original Google pagerank, right? A "good" site linking to another site is what makes that other site some amount of "good" too, links from better sites carry more 'juice'. "good" is of course not just binary, but a quantitative weight. I don't know to what extent that's still at the core of their relevancy rankings. I don't know how all those annoying spammy recipe blogs or content farms get to the top of the results either. I don't think it's because Google's engineers believe they are "good" results. Relevancy ranking on web search is clearly a hard problem, mainly because so many authors are trying to game it, it's a feedback loop. If Google can only do as google does despite pouring a whole lot of money into it, I don't see a reason to bet that better will be what's basically an over-simplified description of how Google started out doing it (and then evolved it because it wasn't good enough).
- JProthero 6y ago
  That was exactly my thought on reading that part of the article: that the author was describing Pagerank, but substituting Pagerank's design objective of algorithmically outsourcing quality judgments to the global community of website publishers with the programmer's prejudices instead. I hope the project of improving on DuckDuckGo is successful though, and some of the other proposals in the article sound promising to me. In the future I would like to see an open source search engine 'paid for' using some combination of homomorphic encryption, blockchain and Tor-style technologies to trade bandwidth and processing power with its userbase in exchange for search results and other services, but I don't have the expertise to assess how feasible that might be.
  - pksebben 6y ago
    the money really is the crux of the issue, too. if you take as a given that consumers won't pay for a service they can get for free, then you're kind of in a bind creating an unopinionated search - if you're truly objective, and reflecting the underlying value of each link faithfully, who do you charge?
mxcrossb 6y ago
As an experiment, I searched “tech news aggregator” on both google and DDG. Neither listed Hacker News. Instead, apart from a few actual sites, most of the links were articles saying “top ten tech news sites” or links to quora q&a threads. It definitely seems that search engines can’t find new websites for people. Now they are just aggregating Q&A.
- bscphil 6y ago
  Yeah - this is sort of my thinking. For better or worse if you enter "discovery" terms, what you will get back is not the results of a "discovery" search, but rather an answer to the question "what are some websites that will help me discover X".
chris_f 6y ago
> For example this is the #2 result for "python f strings" on DDG[1]. It's total garbage, and, quoting the blog, "we can do better". (This result is also on page 1 for the same query on Google.) There are other options that may be better, but in general, very few people are looking for them. Here's your "python f strings" query on Runnaroo: https://www.runnaroo.com/search?term=python+f+strings https://www.runnaroo.com/search?term=python+f+strings I'm the creator of Runnaroo.
- edgyquant 6y ago
  I tried this out real quick just to see for myself. First it was slow for me, if I wasn’t wanting to try it out I may have closed the tab, but I did a search for a question I recently googled (that was an example of an annoying google search.) I searched “Python arcade sprite opacity“ and while the first three results were the same as google the fourth was a github link to the project itself and brought me to the answer (which wasn’t on page 1 of google for me, although it was on page two.) So you need to speed it up, but it does look good for searching documentation at least.

jrochkind1 6y ago

> You do that by having a list of known good sites to answer specific types of questions, and looking at the sites they link to. I mean, that's basically the core of original Google pagerank, right? A "good" site linking to another site is what makes that other site some amount of "good" too, links from better sites carry more 'juice'. "good" is of course not just binary, but a quantitative weight. I don't know to what extent that's still at the core of their relevancy rankings. I don't know how all those annoying spammy recipe blogs or content farms get to the top of the results either. I don't think it's because Google's engineers believe they are "good" results. Relevancy ranking on web search is clearly a hard problem, mainly because so many authors are trying to game it, it's a feedback loop. If Google can only do as google does despite pouring a whole lot of money into it, I don't see a reason to bet that better will be what's basically an over-simplified description of how Google started out doing it (and then evolved it because it wasn't good enough).

JProthero 6y ago
That was exactly my thought on reading that part of the article: that the author was describing Pagerank, but substituting Pagerank's design objective of algorithmically outsourcing quality judgments to the global community of website publishers with the programmer's prejudices instead. I hope the project of improving on DuckDuckGo is successful though, and some of the other proposals in the article sound promising to me. In the future I would like to see an open source search engine 'paid for' using some combination of homomorphic encryption, blockchain and Tor-style technologies to trade bandwidth and processing power with its userbase in exchange for search results and other services, but I don't have the expertise to assess how feasible that might be.
- pksebben 6y ago
  the money really is the crux of the issue, too. if you take as a given that consumers won't pay for a service they can get for free, then you're kind of in a bind creating an unopinionated search - if you're truly objective, and reflecting the underlying value of each link faithfully, who do you charge?

mxcrossb 6y ago

As an experiment, I searched “tech news aggregator” on both google and DDG. Neither listed Hacker News. Instead, apart from a few actual sites, most of the links were articles saying “top ten tech news sites” or links to quora q&a threads. It definitely seems that search engines can’t find new websites for people. Now they are just aggregating Q&A.

bscphil 6y ago
Yeah - this is sort of my thinking. For better or worse if you enter "discovery" terms, what you will get back is not the results of a "discovery" search, but rather an answer to the question "what are some websites that will help me discover X".

chris_f 6y ago

> For example this is the #2 result for "python f strings" on DDG[1]. It's total garbage, and, quoting the blog, "we can do better". (This result is also on page 1 for the same query on Google.) There are other options that may be better, but in general, very few people are looking for them. Here's your "python f strings" query on Runnaroo: https://www.runnaroo.com/search?term=python+f+strings https://www.runnaroo.com/search?term=python+f+strings I'm the creator of Runnaroo.

edgyquant 6y ago
I tried this out real quick just to see for myself. First it was slow for me, if I wasn’t wanting to try it out I may have closed the tab, but I did a search for a question I recently googled (that was an example of an annoying google search.) I searched “Python arcade sprite opacity“ and while the first three results were the same as google the fourth was a github link to the project itself and brought me to the answer (which wasn’t on page 1 of google for me, although it was on page two.) So you need to speed it up, but it does look good for searching documentation at least.