5 ms·

Does anyone remember how back in the day, we had Lycos, Hotbot, Yahoo, Dogpile .. a whole host of search engines. You'd go to two or three to get an idea of wha

Does anyone remember how back in the day, we had Lycos, Hotbot, Yahoo, Dogpile .. a whole host of search engines. You'd go to two or three to get an idea of what was out there. People had different indexes and there were a lot of players. Today, it's just Google. I've been using DDG for a few years, but about 1/3 of the time I add an !g because I don't find the results I need on DDG. The cost of entry to the search market is exceedingly high right now. This is a pretty good article of detailing how one person was able to come up with an idea and challenge the behemoth in very niche areas (privacy/the nsa leaks were probably the reason I started looking at/using it around 2013). Yet I still miss the days of using multiple search engines; seeing a variety of results. I hate the de factor standard of Google. When a company controls that much of search, they get to define the narrative. They literally shape the way many people perceive the world. I wonder if tech will get to the point where indexing will be easier and we'll see more solutions that are cheaper and that can crawl larger datasets with lower processing requirements. Maybe the next step will be distributed search with shared indexes? In any case, Google can't remain on top forever (at least I hope not). It'd be nice to see more tech in this space, but it's an incredibly difficult problem. There is reason Google climbed to the top like it did.

ianai 9y ago
I really like the localized index idea someone posted earlier. If I could get 60% of my searches answered locally for the cost of 100 gigs of space I would use it. Even more so if I could host it on a local server somehow.
- roryisok 9y ago
  Why though? Is your connection that bad? I wouldn't want to use 100GB of precious SSD storage for something I can get instantly in the browser
i_cant_speel 9y ago
While it would be nice if the barrier to entry was lower to enter the search space, I feel like if it's easier to get in Google will just be able to do more with what they have.
tinus_hn 9y ago
Storage, computation and bandwidth costs have never been cheaper. The problem is that Google has an extremely high market share which makes it really difficult to get users. Google climbed to the top because they focused on what matters: the right results, quickly, no fuss. Their competitors gave you unrelated spam for most queries and were slow and focusing on useless things. Googles results are getting worse, there's more fuss than ever and it's getting slower all the time. So there definitely is an opportunity (which DuckDuckGo is taking).
captainmuon 9y ago
+1 > Today, it's just Google. I've been using DDG for a few years, but about 1/3 of the time I add an !g because I don't find the results I need on DDG. If DDG folks are reading this, here is an idea: Have a browser extension (so opt-in). If a user then does a !g search, forward the search terms and the search results to DDG and add them to your index. (I wouldn't go as far as Microsoft, who IIRC actually copied the results and the ranking for individual terms and added them to Bing.)

ianai 9y ago

I really like the localized index idea someone posted earlier. If I could get 60% of my searches answered locally for the cost of 100 gigs of space I would use it. Even more so if I could host it on a local server somehow.

roryisok 9y ago
Why though? Is your connection that bad? I wouldn't want to use 100GB of precious SSD storage for something I can get instantly in the browser

i_cant_speel 9y ago

While it would be nice if the barrier to entry was lower to enter the search space, I feel like if it's easier to get in Google will just be able to do more with what they have.

tinus_hn 9y ago

Storage, computation and bandwidth costs have never been cheaper. The problem is that Google has an extremely high market share which makes it really difficult to get users. Google climbed to the top because they focused on what matters: the right results, quickly, no fuss. Their competitors gave you unrelated spam for most queries and were slow and focusing on useless things. Googles results are getting worse, there's more fuss than ever and it's getting slower all the time. So there definitely is an opportunity (which DuckDuckGo is taking).

captainmuon 9y ago

+1 > Today, it's just Google. I've been using DDG for a few years, but about 1/3 of the time I add an !g because I don't find the results I need on DDG. If DDG folks are reading this, here is an idea: Have a browser extension (so opt-in). If a user then does a !g search, forward the search terms and the search results to DDG and add them to your index. (I wouldn't go as far as Microsoft, who IIRC actually copied the results and the ranking for individual terms and added them to Bing.)