5 ms·
I wish them success, but honestly I'm just curious around how a search startup (and I mean a true search startup, one that does its own indexing) can even hope
by hn_throwaway_99 6y ago
I wish them success, but honestly I'm just curious around how a search startup (and I mean a true search startup, one that does its own indexing) can even hope to compete against the giants these days (and let's face it, in the English language it's pretty much all Google with Bing a distant second).
I mean, when Google started the internet was actually small enough that it could be indexed relatively cheaply. These days, the internet is so enormous, and user expectations around hyper local content and being constantly up to date are so high, I don't see how one could begin to compete without billions.
- gkoberger 6y agoLooks like their goal isn't to compete with Google on volume, but something else. It's a paid search, and it's ad-free, so I assume they're targeting people who care about privacy. I don't think they believe they'll surpass Google. But if they can get 0.05% of Google's users to pay them $5/month, they're making $120 million a year. That's a great business by most normal standards. (Now, I don't think this will happen, necessarily... but I understand why you'd make that bet)
- nikanj 6y agoHey I just pointed out this 0.05% idea in another comment! It really is a common sentiment amongst investors that you can just fling shit towards a market giant and hope to capture a fraction of their market share. At FAANG scales even a fraction of a percent is big bucks.
- ssss11 6y agoHmm.. the ex-Head of Ads providing an ad-free privacy-first search engine? Not sure I’d trust them...
- gkoberger 6y agoI mean, counter argument is that nobody knows how shitty and toxic ads are as he does. Google was just a job and I doubt he was ever enthusiastic about ads. It’d be like someone who worked at a meat packing factory going vegan.
- peteretep 6y ago> I'm just curious around how a search startup ... can even hope to compete If you held a gun to my head and told me to build a Google competitor, I'd start off with smaller products with shared search infrastructure. Start off with some hyper-focused projects that you can actually compete with Google for: code search, product search, news article search, review search, search focusing on kids, etc. Build a better YouTube search and pay a legal team to force Google to let you have some access to it. Maybe a better and free "search my site" functionality. Make some of them good enough the people just use them, get Bing or DDG interested in supplimenting their results with yours. Have these as separate brands that people know, but for every search have a button that takes you to the wider search and start building brand recognition.
- loosetypes 6y agoIs there a good YouTube search out there? say you’re trying to find a ChrisStuckmann review for a particular movie. Sometimes even when you explicitly include a channel’s name in a search their magic excludes it from results. The same query from within YouTube search and from Google search can lead to different results. And YouTube’s search within a channel from that channel’s page is even worse.
- aynsof 6y agoGoogle's search is appalling these days. If I search for three terms, I have to include "each" "of" "them" in quotation marks, or else Google assumes I didn't actually want to search for them. It's incredibly frustrating when the front door to the internet doesn't actually do the one thing it's supposed to do.
- Ueland 6y agoAnd the amount of spam in search results now is even more appalling. Pintrest + translated content is the first two things that comes to mind.
- ehnto 6y agoGoogle can't even solve the Pinterest image search problem, which has gone on for years. I think they are becoming the incumbent too heavy to move under it's own technology platform, or unwilling to change. Even if you gleefully state that the algorithm finds what users most want, Google is a search product not a technology art installation, and what user's engage with is a function of what you show them in the first place. Pinterest should not be 50%+ of the results for any image search. Pinterest is not the greater internet.
- daniellarusso 6y agoYeah, this is a really bizarre one, in my opinion. Google indexes and provides results for many sites whose content is inaccessible without an account, yet these results take precedent over actual unrestricted content. Similarly with Google News, why are they showing me headlines of articles I am not able to access?
- zpeti 6y agoI'm starting to think over 50% of google's magic is trying to actually get rid of the spam. I think a new search engine would have a very hard time getting through challenges google has already, like link spam, authority domain spam (using medium etc for spammy content), and many many other issues about people manipulating results.
- snowwrestler 6y ago
- hunter-2 6y ago>I mean, when Google started the internet was actually small enough that it could be indexed relatively cheaply. Infrastructure is much cheaper today than it was in 1998. Anybody with the tech know-how can start a mini search engine from their bedroom today without breaking the bank. I see Neeva as the search-equivalent of Hey. Both products compete against a free product that Google dominates in, with a subscription alternative. Time will tell if there really is a market for this model.
- kybernetikos 6y agoI think there's lots of room for something interesting here. Partly because google results aren't great at the moment and haven't been great for quite a while and partly because the internet is more balkanised than it was, and often I'd be happy to keep my queries within one of the partitions. For example, search targetting just the websites that don't sell my data to 300 third parties, or that are available worldwide (not just outside the eu) or search that targets just a curated set of websites that are making an effort to provide authoritative content (e.g. wikipedia, stack overflow, respected news sources, actual academic papers). For searching for things I've recently discovered, search that spiders from aggregators and social media (reddit, hackernews, twitter, facebook) would be most valuable. Search inside audio (podcasts), video and images could also be much better. If I could bring myself to trust them, I'd want something where every site I visited was spidered and I could do full text search within my own history. The long tail stuff is hard, but I think it'd be OK to delegate that stuff to currently existing engines, you could provide massive value just by doing a better job of the basics. Point being, I think the space has quite a lot of potential for disruption despite the challenges.
- weinzierl 6y agoI agree with you in that I cannot imagine a search startup to successfully compete with Google in the current situation, but for different reasons: I think the index is not the biggest challenge. Yes, the internet was small when Google started and is enormous now, but there has also been a lot of concentration in the sense that the most requested content is hosted on fewer and fewer sites. Also, the expectations changed. In the beginning, the idea was to index the whole web, and search engines bragged about the size of their index. Nowadays, it's all quality over quantity, and index size is probably not the most critical factor. Apart from that: Building a decently sized index is a major undertaking, but it is not impossible. While the web has grown, technology also has advanced, and there are multiple companies that actively maintain a search engine scale index. SEO companies, like ahrefs or majestic, do it for their purposes, and there is also Common Crawl. The real challenge, I believe, is that Google has access to a ton of engagement data. Google Analytics, AdSense, Google Fonts, Google Public DNS, and probably other services provide Google with excellent and hard to manipulate metrics about sites' popularity. To successfully compete with Google, a startup would have to find an ethical alternative for this. I believe that building on unethically sourced engagement data from Browsers and Browser extensions will backfire in the long run. The Brave search engine is an example of this. The best chance to succeed here have the large CDN companies, like Cloudflare, because they could harvest some of the data in a non-invading way. > [..] and let's face it, in the English language, it's pretty much all Google with Bing a distant second I'd argue that the non-English world is, even more, all Google because it is the only search engine that works appropriately for non-English queries as well as for local search results. Baidu in China is the only real exception to this. Even in Russian-speaking countries, Google's market share is higher than their (decent) local search engine Yandex.
- llaolleh 6y agoIf you read the original Anatomy of a Search Engine paper by Google co-founders, you'd be amazed at the constraints they had to engineer around with back then. Many of the constraints that they had to engineer around don't exist anymore. The landscape has changed significantly since 1998. The decrease in bandwidth costs, increase in computing power, RAM, infrastructure as a service, and the rapidly moving ML research that pops out new NLP techniques. You also have to consider Paul Graham's advice on working backwards. Google is nowhere near the final form of information retrieval. In an ideal world, I would have an answer to a search query instantaneously. Even we solve that problem, then you begin to try to solve the problem of why you even had to query for information in the first place.
- austhrow743 6y ago>user expectations around hyper local content For a general search engine yes. Less so for a specialty one. If I'm searching for javascript documentation I don't care if it's attached to a domain attached to a business with a physical location near me. Imo if a search startup appears it'll be through a tiny niche -> larger niche -> even larger niche -> general search, path. Not nothing -> general search.
- rstuart4133 6y agoThe other day the sole man behind this search engine popped up here on HN: https://gigablast.com/ https://gigablast.com/ He apparently does his own spidering and own indexing. Absurd, right? I did a few tests. It's not as fast, it's definitely not as good for current affairs (ie, very recent stuff) and the author says it's not so good with synonyms. But outside of that it was hard to fault, so as DDG (well bing's) results are annoyingly uneven I switched to using it as my default. I didn't realise how lazy and dependent I've become on Google and Bing's ability to guess what I meant from the spelling mistake ridden crap I entered. If I put those quibbles aside, I'd rate it better than DDG. Which I find extraordinary. For now it remains my default.
- deleted 6y ago[deleted]
- gyulai 6y ago> I mean, when Google started the internet was actually small enough that it could be indexed relatively cheaply. These days, the internet is so enormous, and user expectations around hyper local content and being constantly up to date are so high, I don't see how one could begin to compete without billions. But you're dealing with insanely long-tailed distributions and the "meat" of the search engine business is in the fat heads of those distributions. (1) A small proportion of queries makes up a huge proportion of query events that you'll see through the day. (2) For any given query, a small proportion of users will make up a huge proportion of the opportunity to monetize (researching a planned purchase, looking for a job etc). (3) For any given query, an infinitesimally tiny proportion of the documents on the web is where the value to the user actually is. I think there are potentially many ways of making selections on each of those three axes and ending up with a viable business based on a manageably small search index E.g. indeed as a job search engine or amazon as a product search engine have manageably small document collections, great value to users and a stable user base, and great opportunities to monetize. ...from that point of view I find it surprising that there aren't a lot more search engine businesses. Case in point: I really like wiby.me, the 90s nostaliga search engine. I don't think they are a profitable business, but I certainly think they could & should be. And even Google is making very deliberate choices along those three dimensions, rather than naively indexing the web and naively executing keyword search against that index. Dimension 1: Given a query, reinterpret as follows. Catch eyeballs by delivering entertainment value ("pizza" -> "entertaining videos related to pizza"). Monetize those eyeballs by reinterpreting as local queries ("pizza" -> "restaurants near me wanting to sell me pizza") or products queries ("pizza" -> "online shops trying to sell me pizza ovens or other pizza-related items") Dimension 2: Given a query and document, always make relevance decisions on behalf of the kind of audience with disposable income. E.g. "farming" shows a lot of stuff you'd want to read if you're the kind of person paying £10 for a potato in Borough Market and nothing that you'd want to read if you're a subsistence farmer in Namibia. Dimension 3: When comparing two documents for inclusion in the pool of documents that stand any chance of coming up on page #1, prefer recency over authoritativeness. E.g. for "programming languages", apparently "The 9 Best Programming Languages to Learn in 2021" is considered relevant while "Go To Statement Considered Harmful" or the Mozilla Developer Network are considered irrelevant. There are huge audiences who don't agree with these choices that Google is making and who are just waiting to switch if someone comes along making better choices or using another product alongside Google if someone comes along simply just making different choices that are useful in some way. So, again, far from the sentiment that any venture in the web search space is doomed given the size of the web and the existence of Google, I'm positively baffled by the fact that there isn't a lot more activity there.