3 ms·
So, finally a paid search engine that does not have to rely on ads for revenue. So far it's based on Bing, which does. This makes it a bit a hard sell, compar
by nine_k 6y ago
So, finally a paid search engine that does not have to rely on ads for revenue.
So far it's based on Bing, which does. This makes it a bit a hard sell, compared to an intelligent ad blocker.
The most important problem of search engines is SEO spam. Google themselves sort of have a moral hazard to not be too stringent on SEO spam, because it shows ads by Google, increasing Google's revenue.
OTOH I wonder if the subscription revenue is going to be sufficient to have access to a reasonably good search index and enough processing power to efficiently combat SEO spam while returning relevant results. This takes your own data centers run frugally, because fees of something like AWS or Azure will just be exorbitant for a global search engine and a global search index.
I wonder if companies aiming to provide alternative search engines will cooperate on maintaining a common index, to distribute the massive costs of doing that. They could even publicly sell access to it, at a point where running a competing search engine won't be practical; e.g. researchers would buy it.
- ma2rten 6y agoI am actually wondering if search engines really need such a large index. The vast majority of sites in Google's index are crap. If you were able to better select which sites to index it might help search quality and improve efficiency.
- nine_k 6y agoHow can you tell what is crap while crawling, is there enough time? How can you tell what is crap for the next person, and what is not crap? It's not an easy question.
- nwienert 6y agoPeople! Let the users help. All the best content sources on the internet are either experts or communities with voting / submissions.
- sah2ed 6y ago> People! Let the users help. All the best content sources on the internet are either experts or communities with voting / submissions. There are over 1.7 billion websites [0], so the task of ranking content, the way algorithmic search engines do it in a matter of milliseconds, is not as easy as it sounds when you add humans into the mix. It would only end up the way Mahalo did [1]. [0] https://www.statista.com/chart/19058/how-many-websites-are-there/ https://www.statista.com/chart/19058/how-many-websites-are-t... [1] https://en.wikipedia.org/wiki/Mahalo.com https://en.wikipedia.org/wiki/Mahalo.com
- nwienert 6y agoFalse dichotomy, you can use algorithms in tandem. And cherry picking... I remember Mahalo well, and one example of one version of it doesn’t prove anything. Mahalo is far from how I’d structure it. You can still automatically index but have users vote on the results. There are 1.7 billion websites and 3 billion+ users, and you don’t need that many to be active voters to help assist algorithms. Plus how many are at the top anyway? I’d love to downvote a ton of google results even if it only used it as a trainer for my own. Plus, there are so many “super curation” sites like here and Reddit that provide a big dataset curated by people automatically. Lean on them more. Everyone knows “site: reddit.com” or “site:stackoverflow.com” already give you better results. A simple upvote downvote on their results would let me downvote all the spam SEO sites. It wouldn’t take many votes for them to start tuning it. Stats are a good way to blind yourself. That algorithms scale doesn’t mean people don’t improve them. Google’s problem is they are too cocky about algorithms, but their algorithms fail compared the curated communities all over the web already.
- asddubs 6y agothe other day I was searching "<sitename> feature" and from the results (which to be fair aren't many, since it was a fairly obscure site), once you get through the legit results on the site i was searching for, there were a bunch of markov generated spam things that had clearly scraped posts from various sites and tangled them together
- BitwiseFool 6y agoThis is just a hunch but I imange the internet also follows the pareto principle and at least 80% of everything in google's index is basically worthless. We may end up with search engines that use multiple indices where each is curated to a certain domain of information, rather than a one-size-fits-all index.
- a3camero 6y agoI'm working on a search engine like this. So far just government websites but I think you're right about the need to focus on selected sites for some kinds of searches. For example, if you're looking for securities laws, you probably want to search government law sites and securities regulator sites. Maybe you want law blogs and law journal articles too, but you probably don't want Reddit or Hacker News.
- bananaface 6y agoI think Google bases a lot of its ranking on bounce rate, i.e. "how often people who searched this were happy with this result." I don't think you need that many hits to establish bounce rate for quite specific searches. Like, is 40 is enough? I can imagine so. But if you filter for paid members, you've filtered for much smarter people (on average) picking the best results - and you've also filtered out a lot of the SEO people who are going to be trying to manipulate the bounce rate with proxies.