5 ms·

I think the utility of generic search engines is coming to an end. Services like Google, Bing and DDG are numbered in usefulness. Instead, I am guessing (maybe

I think the utility of generic search engines is coming to an end. Services like Google, Bing and DDG are numbered in usefulness. Instead, I am guessing (maybe hoping) that we see a return of the moderated directories, like Yahoo or DMOZ of old. A 2.0 spin on these directories, with lessons learned from all the years. Imagine that you just go to StackOverflow and search there directly for your answer. Want a funny laugh, go to your favorite comedy website (like facebook.com) and search directly. Want some news, go to CNN or Fox (depending on your persuasion) and fill your echo chamber. Not kidding. I think search as we know it is dead. And there is a new paradigm just sitting out there ready for the next generation to make. A directory of well known lists of websites with specialty search engines available on each.

eawoifjaiowepfj 5y ago
i just use google to search reddit these days.
- rhizome 5y ago
  Funny, Google search is so cluttered these days I almost automatically append "reddit" to any product or technique search. I think a lot of people are doing this, because the "People also ask" block on search results will often have questions like "best lawnmower reddit?" (a fine example of that Google quality we've all heard so much about)
- mateo1 5y ago
  Which is why reddit is starting to flood with spam. It won't last long.
- waynesonfire 5y ago
  Sshh, you're ganna ruin it.
ravenstine 5y ago
I've been coming to a similar conclusion. Increasingly I've had to use `site:<domain name>` syntax with DDG, but DDG has clearly become significantly worse in recent months. This was confirmed by others in another HN thread recently. Even if I use the site-syntax or use quotes to specify exact match (which DDG used to honor), I'll often get nonsense results whereas Yandex somehow manages to find results that aren't on DDG or Google. And yeah, I've been having to use search engines on specific sites like Wikipedia because I know there's gonna be less shenanigans. Sometimes I wonder if I'm living in the past. Do normies actually use search engines anymore?
- taftster 5y ago
  I think there's a market for sites that have optimized search around a specific category of information. For example, imagine you had a hybrid directory+search model, where you could drill down into the directory: Top -> Programming -> Lisp -> Scheme And from there, search a repertoire of Scheme related articles, tips, blogs, etc. This model already exists, somewhat, in the form of scholarly articles, medical journals, etc. But there seems to be some innovation needed. If there were such a "site" that focused on and aggregated Scheme related documentation, you'd think that being listed on that search-directory would be totally acceptable to the website owner. After all, a web article or blog generally wants to be read, especially by an interested reader. Somehow we need to get back to the web being like a library, with a librarian and a well organized categorical index for references. Getting humans back into the search loop I think is where we're going to circle to. Like asking your local librarian for information about a topic that is in their special collection.
  - bryanrasmussen 5y ago
    this is known as faceted search https://en.wikipedia.org/wiki/Faceted_search https://en.wikipedia.org/wiki/Faceted_search on edit: I remember Carrot2 had the ability to derive categories from an index dynamically at one point, so when we put all the Danish laws into it and you searched for murder it generated categories such as Knife Murder etc. anyway here is the online Carrot2 with a search on Scheme https://search.carrot2.org/#/search/web/scheme/folders https://search.carrot2.org/#/search/web/scheme/folders on edit2: obviously this Carrot2 online thing is just a demo, they don't have a particularly big or up to date index. I'm also not sure how well Carrot2 would scale as it was, IIRC, an academic project to investigate different ways of searching / organizing search results. Check out the treemap.
    - taftster 5y ago
      Hey - thanks for giving a name for this and the link to Wikipedia. I know it's not a "new" idea, but a faceted search organized and usable broadly across the www would be really useful.
- 5y ago
ireadfaces 5y ago
I tried that, but I try to search in wide world, sometimes about topics that I don't even understand. So I don't know where to look. And many of my searches are like that (examples: how merchant fees work, how QC affects stock prices, effects of vitamin D3, how price parity is determined, how to focus on something boring) All these are varied terms and most of search results will be like that, so I wonder if moderated directories would ever serve that.
- taftster 5y ago
  I hear you. But arguably, imagine you could search a "collection" after you drilled down into your category. Top -> Finance -> Credit Card -> Merchant Fees + Search Top -> Finance -> Stock Market -> Algorithms + Search Top -> Health -> Vitamins + Search And of course, the Wikipedia is a good jumping off point for all of that information. But imagine a wikipedia-like place that then allowed you to find the top level information that you're looking for with additional search capabilities in its supporting collection. Maybe the start of this is taking a Wikipedia article on Price Parity and then enumerating the referenced articles in the footnotes and putting a search engine over those perhaps. I don't know. But there's got to be something better than what we have right now. I feel another leap of search innovation happening soon.
- mateo1 5y ago
  You'll have to think for yourself a bit. Which platform and country are the merchant fees about? Do you want the official source and legal text, a professionals opinion from a magazine or newspaper, or a blog post by someone detailing what they experienced? A good search engine can and should ask you about this and present you with your choices. Or, of course, they can choose what to show you to optimise for something, be it response time, resource consumption or ad profit.
twobitshifter 5y ago
How many of your searches lead directly to Wikipedia or stackoverflow or GitHub. Often we know where we should go but the default search engine in the address bar leads us through a Google intermediary. I myself use keyword searches to go directly to sites, but I doubt that is very common.
PixyMisa 5y ago
Try DuckDuckGo. It has shortcuts for searching - at last count - 13,565 sites.
nmstoker 5y ago
But is this driven by the supply side or demand side? Seems to me that users would still benefit from a cross-site search capability (eg to follow your example, what if you want something funny not from a specific site or you don't know which sites offer something) It's subjective but most of the pressure seems to be because the search quality has dropped off recently, not that I'm no longer willing/eager to search.
Barrin92 5y ago
isn't that pretty much what you.com tries to do? Sort of a combination of both ordinary search results and categories based on sources.
pengaru 5y ago
Such optimism! Don't you think it's more likely in a post keyboard/computer-literate world Search web sites are displaced by vendor-provided App search boxes? You wanted news? Here's the CNN proprietary app in this beautiful walled garden! Enjoy the ads and total loss of privacy! [This app requires the following permissions: ALL YOUR BASE ARE BELONG TO US]
- taftster 5y ago
  I think that's a fair point. The world of "apps" has certainly changed the meaning and usefulness of search. There's no exploring and random discovery when you're in the walled garden, unlike what it used to be like when you could really "surf" the web.
beprogrammed 5y ago
I hope to see the rise of distributed search. It's not that generic search is the problem, just that companies have different, and often counter, priorities than users.
a5aAqU 5y ago
Internal search can often be scammy though. If you search Google for products, you'll land on an Amazon page, but if you do your search on Amazon instead, they will sometimes only show you more expensive versions of the general product, as if the cheap one that you were looking for doesn't exist.
colordrops 5y ago
I don't buy it. The technology to fight SEO gaming and filter bad results has been around for a while and is only getting better, but google is getting greedy and sloppy. Google has mind-blowing AI/machine learning capabilities and I didn't see shit about cleaning up search results in their last update. They could fix this if they wanted, and an upstart company that figures it out will eat google's lunch. Google makes an insane amount of profit, and a search engine that actually works may make less but would still be profitable.
webmaven 5y ago
> Instead, I am guessing (maybe hoping) that we see a return of the moderated directories, like Yahoo or DMOZ of old. A 2.0 spin on these directories, with lessons learned from all the years. Not impossible, but given the proliferation of content on the web and the endless possibilities for competing organizing schemas and editorial choices, it would pretty much have to be a directory of directories. So, each directory could be it's own self-organizing community, like a subreddit.
NetOpWibby 5y ago
You should try Neeva[0], made by ex-Google Search folks. [0]: https://neeva.com/p/signup?o=s&c=c3af6jnfmc2dehs8tq5g https://neeva.com/p/signup?o=s&c=c3af6jnfmc2dehs8tq5g
- black_puppydog 5y ago
  "coming soon to your region" Ugh, that triggers me hard. I haven't lived in Germany nor Sweden for ~7 years now and Google never really figured out (despite me telling it so) that I don't need gmaps results in Stockholm and Berlin, and that yes, all my g products should be showing the English UI. If these folks are ex-google they might consider not doing shit like this in the first place.
  - NetOpWibby 5y ago
    Damn, that sucks! No idea they did things like that.

eawoifjaiowepfj 5y ago

i just use google to search reddit these days.

rhizome 5y ago
Funny, Google search is so cluttered these days I almost automatically append "reddit" to any product or technique search. I think a lot of people are doing this, because the "People also ask" block on search results will often have questions like "best lawnmower reddit?" (a fine example of that Google quality we've all heard so much about)
mateo1 5y ago
Which is why reddit is starting to flood with spam. It won't last long.
waynesonfire 5y ago
Sshh, you're ganna ruin it.

ravenstine 5y ago

I've been coming to a similar conclusion. Increasingly I've had to use `site:<domain name>` syntax with DDG, but DDG has clearly become significantly worse in recent months. This was confirmed by others in another HN thread recently. Even if I use the site-syntax or use quotes to specify exact match (which DDG used to honor), I'll often get nonsense results whereas Yandex somehow manages to find results that aren't on DDG or Google. And yeah, I've been having to use search engines on specific sites like Wikipedia because I know there's gonna be less shenanigans. Sometimes I wonder if I'm living in the past. Do normies actually use search engines anymore?

taftster 5y ago
I think there's a market for sites that have optimized search around a specific category of information. For example, imagine you had a hybrid directory+search model, where you could drill down into the directory: Top -> Programming -> Lisp -> Scheme And from there, search a repertoire of Scheme related articles, tips, blogs, etc. This model already exists, somewhat, in the form of scholarly articles, medical journals, etc. But there seems to be some innovation needed. If there were such a "site" that focused on and aggregated Scheme related documentation, you'd think that being listed on that search-directory would be totally acceptable to the website owner. After all, a web article or blog generally wants to be read, especially by an interested reader. Somehow we need to get back to the web being like a library, with a librarian and a well organized categorical index for references. Getting humans back into the search loop I think is where we're going to circle to. Like asking your local librarian for information about a topic that is in their special collection.
- bryanrasmussen 5y ago
  this is known as faceted search https://en.wikipedia.org/wiki/Faceted_search https://en.wikipedia.org/wiki/Faceted_search on edit: I remember Carrot2 had the ability to derive categories from an index dynamically at one point, so when we put all the Danish laws into it and you searched for murder it generated categories such as Knife Murder etc. anyway here is the online Carrot2 with a search on Scheme https://search.carrot2.org/#/search/web/scheme/folders https://search.carrot2.org/#/search/web/scheme/folders on edit2: obviously this Carrot2 online thing is just a demo, they don't have a particularly big or up to date index. I'm also not sure how well Carrot2 would scale as it was, IIRC, an academic project to investigate different ways of searching / organizing search results. Check out the treemap.
  - taftster 5y ago
    Hey - thanks for giving a name for this and the link to Wikipedia. I know it's not a "new" idea, but a faceted search organized and usable broadly across the www would be really useful.
5y ago

ireadfaces 5y ago

I tried that, but I try to search in wide world, sometimes about topics that I don't even understand. So I don't know where to look. And many of my searches are like that (examples: how merchant fees work, how QC affects stock prices, effects of vitamin D3, how price parity is determined, how to focus on something boring) All these are varied terms and most of search results will be like that, so I wonder if moderated directories would ever serve that.

taftster 5y ago
I hear you. But arguably, imagine you could search a "collection" after you drilled down into your category. Top -> Finance -> Credit Card -> Merchant Fees + Search Top -> Finance -> Stock Market -> Algorithms + Search Top -> Health -> Vitamins + Search And of course, the Wikipedia is a good jumping off point for all of that information. But imagine a wikipedia-like place that then allowed you to find the top level information that you're looking for with additional search capabilities in its supporting collection. Maybe the start of this is taking a Wikipedia article on Price Parity and then enumerating the referenced articles in the footnotes and putting a search engine over those perhaps. I don't know. But there's got to be something better than what we have right now. I feel another leap of search innovation happening soon.
mateo1 5y ago
You'll have to think for yourself a bit. Which platform and country are the merchant fees about? Do you want the official source and legal text, a professionals opinion from a magazine or newspaper, or a blog post by someone detailing what they experienced? A good search engine can and should ask you about this and present you with your choices. Or, of course, they can choose what to show you to optimise for something, be it response time, resource consumption or ad profit.

twobitshifter 5y ago

How many of your searches lead directly to Wikipedia or stackoverflow or GitHub. Often we know where we should go but the default search engine in the address bar leads us through a Google intermediary. I myself use keyword searches to go directly to sites, but I doubt that is very common.

PixyMisa 5y ago

Try DuckDuckGo. It has shortcuts for searching - at last count - 13,565 sites.

nmstoker 5y ago

But is this driven by the supply side or demand side? Seems to me that users would still benefit from a cross-site search capability (eg to follow your example, what if you want something funny not from a specific site or you don't know which sites offer something) It's subjective but most of the pressure seems to be because the search quality has dropped off recently, not that I'm no longer willing/eager to search.

Barrin92 5y ago

isn't that pretty much what you.com tries to do? Sort of a combination of both ordinary search results and categories based on sources.

pengaru 5y ago

Such optimism! Don't you think it's more likely in a post keyboard/computer-literate world Search web sites are displaced by vendor-provided App search boxes? You wanted news? Here's the CNN proprietary app in this beautiful walled garden! Enjoy the ads and total loss of privacy! [This app requires the following permissions: ALL YOUR BASE ARE BELONG TO US]

taftster 5y ago
I think that's a fair point. The world of "apps" has certainly changed the meaning and usefulness of search. There's no exploring and random discovery when you're in the walled garden, unlike what it used to be like when you could really "surf" the web.

beprogrammed 5y ago

I hope to see the rise of distributed search. It's not that generic search is the problem, just that companies have different, and often counter, priorities than users.

a5aAqU 5y ago

Internal search can often be scammy though. If you search Google for products, you'll land on an Amazon page, but if you do your search on Amazon instead, they will sometimes only show you more expensive versions of the general product, as if the cheap one that you were looking for doesn't exist.

colordrops 5y ago

I don't buy it. The technology to fight SEO gaming and filter bad results has been around for a while and is only getting better, but google is getting greedy and sloppy. Google has mind-blowing AI/machine learning capabilities and I didn't see shit about cleaning up search results in their last update. They could fix this if they wanted, and an upstart company that figures it out will eat google's lunch. Google makes an insane amount of profit, and a search engine that actually works may make less but would still be profitable.

webmaven 5y ago

> Instead, I am guessing (maybe hoping) that we see a return of the moderated directories, like Yahoo or DMOZ of old. A 2.0 spin on these directories, with lessons learned from all the years. Not impossible, but given the proliferation of content on the web and the endless possibilities for competing organizing schemas and editorial choices, it would pretty much have to be a directory of directories. So, each directory could be it's own self-organizing community, like a subreddit.

NetOpWibby 5y ago

You should try Neeva[0], made by ex-Google Search folks. [0]: https://neeva.com/p/signup?o=s&c=c3af6jnfmc2dehs8tq5g https://neeva.com/p/signup?o=s&c=c3af6jnfmc2dehs8tq5g

black_puppydog 5y ago
"coming soon to your region" Ugh, that triggers me hard. I haven't lived in Germany nor Sweden for ~7 years now and Google never really figured out (despite me telling it so) that I don't need gmaps results in Stockholm and Berlin, and that yes, all my g products should be showing the English UI. If these folks are ex-google they might consider not doing shit like this in the first place.
- NetOpWibby 5y ago
  Damn, that sucks! No idea they did things like that.