21 ms·
Honestly I think people are completely underestimating the difficulty of a good search engine. Google was better ten years ago because search was an easier prob
by fdgsdfogijq 5y ago
Honestly I think people are completely underestimating the difficulty of a good search engine. Google was better ten years ago because search was an easier problem to solve back then. End of story. Nobody is coming along with a better search engine. There is too much spam, content gaming, and money to be made by hacking search.
These posts should almost be blocked from hacker news. ITs a fantasy. Its like saying that democracy has failed so lets replace it, replace with what? Its the best we can get given the alternatives, and its flaws will always be exploited.
- taysix 5y agoNot sure if you read the post. They don't advocate for replacing Google. They want to add onto Google and other search engines.
- joering2 5y agoDemocracy would be best replaced with cashless society. Society based on everyone value of doing what they love doing most (aka hobbyist), and virtually everything else replaced by robots and technology we already have. This is what's coming eventually, but its not something that can be installed on the top of your "operating system"; you will have to format the whole hard-drive (civil war)
- xboxnolifes 5y agoI don't understand how any of that has anything to do with democracy.
- FastMonkey 5y agoI suppose in the absence of cash we would just force some people to do the necessary tasks that aren't anyone's favourite hobbies?
- xerox13ster 5y agoWell, ideally people could choose to do whatever they liked to provide for society. Everyone would have a role to play. So those that like to farm (raised on a farm, to be proud of their work) would farm, those that want to commit their life to science could attempt to advance their field. Someone might find an invigorating pride in making sure wastewater treatment is running well, but they don't know how to get into the field and are stuck working a job they don't like for money, currently. Artists could create culture. As long as you are doing something to provide for society, society will provide for you. There will be people that do not have favorite hobbies. There will be people with no ambition who benefit from being told what they can or should do to contribute. Those people, in order to have their needs met could choose to take on a Necessary Task, even if it isn't their favorite thing. Maybe they learn to love it and excel at it. Don't want to do anything to contribute to society despite being physically and mentally able? No needs met. Meet them yourself, you magnificent lone wolf. Want your needs met but don't want to do Necessary Task that you hate? Find a purpose and pursue it.
- joering2 5y agoCouldn't have written it better myself. Thank YOU!
- FastMonkey 5y agoI don't know how to break it to you, but the system you're looking for that does all the things you're saying is capitalism. Have the talent and desire to be a heart surgeon? Go for it, and you'll be rewarded. Want to make life sized statues of the Beatles out of used printer cartridges? Eh, fine but maybe also get a day job.
- JohnHaugeland 5y agoIt's sort of weird how badly people have become unable to understand what the word capitalism actually means, these days It seems like everyone just thinks it's how you describe the contemporary world No, what they're describing isn't capitalism. Not even close
- xerox13ster 5y ago
- zargon 5y agoNo. Google is an actively hostile experience. Try it without adblock and get a taste of how most people get treated by Google. And for the search results, Kagi is already better for 90% of my queries. It's much more difficult now to build a competitive search engine, but saying it's impossible and discussions should be banned is toxic. (And already basically proven wrong with existing competing search engines.)
- fdgsdfogijq 5y agoI think that with current technology, beating google at the scale they run at is impossible. I think with advances it NLP, its possible. But right now, its pie in the sky.
- Kuinox 5y agoI disagree, Kagi is not yet "better in 90% of the queries" than google, it's good enough to not have to launch google in 90% of it's query.
- zargon 5y agoThis is probably true for me, if we also say that 50% of the queries are orders of magnitude higher quality on Kagi.
- omnicognate 5y agoKagi's better than Google for me. Proof is that initially I only had it set up as default search on one device but I've progressively switched them all over out of annoyance with crap google results. I haven't yet had to go to google once to finish a search I started on Kagi.
- lolinder 5y agoIt's substantially better than Google for what I do with it (mostly work-related queries). A few reasons why: * It blocks all the GitHub and Stack Overflow copycat sites out of the box. * I can add additional sites to the block list with two clicks. * It up-ranks (semi-)official documentation like the Python docs or MDN over blog spam sites. * I can manually up-rank sites (and even pin them to the top of the results) if I'd like to see them more often. Kagi is awesome because where Google thinks it can guess what you want, Kagi gives you an easy way to tell it what you want.
- bsder 5y ago> Honestly I think people are completely underestimating the difficulty of a good search engine. No, they are underestimating the difficulty of funding a good search engine. I liked the runnaroo search very much as did several of my friends. The guy who ran it couldn't fund it. He shut it down. Altavista (Yeah, that far back) had a nice feature where it would draw a cluster graph of your search results. So, if you searched for "python", it would show your results but would also draw a little graph and you could see that "Hey, there are two clusters here--programming and reptiles." You could then click on the "programming" node and the "reptiles" cluster would go away. It allowed you to drill through irrelevant stuff really quickly. Note how that feature doesn't exist today--in spite of orders of magnitude more programmers being thrown at search, graph algorithms, and nifty Javascript web UIs. I wonder why ... (/sarcasm in case you missed it. I don't wonder why. Such a feature would let you drill through irrelevant Ad and SEO garbage too quickly and would impact Google's revenue.)
- aghilmort 5y agobuilding Breeze, which does pretty much that minus the cool UI angle, and yeah, only way for it to work at scale is premium like neeva, kagi, etc. we started out with topics and then moved to web, and are now folding topics into the web search experience, it's really hard stuff to get right our first main filter was blogs, which is getting renamed to posts for a mix of reasons, adding forums shortly, along with other more specific topics re: https://breezethat.com/ https://breezethat.com/ -- & better on laptop / desktop atm, premium version will be ad-free
- thomassmith65 5y agoHonestly I think people are completely underestimating the difficulty of a good search engine I suspect that is not difficult so much as expensive. Boutique search engines pop up all the time here on HN, but they can't compete fairly against Google, without the resources to crawl a billion webpages day after day.
- charcircuit 5y agoGoogle also keeps a copy of the "entire" web in RAM to search it faster.
- 1_player 5y agoSource? But it's definitely possible nowadays when RAM is cheap and modern Google makes the Internet feel incredibly smaller than it was 20 years ago. At the time you would get tons of results from a myriad of small blogs, forums, niche websites. Nowadays it's Pinterest, blogspam and more SEO optimised algorithmic crap. If you're lucky you get a forum result that might actually be relevant. Given the joke that if you can't Google it it doesn't exist, it follows that the Internet has considerably shrunk in the past two decades.
- charcircuit 5y agoIt looks like I was mistaken. They don't keep all of the documents in RAM, but I'm pretty sure they keep the search index (100 petabytes) in RAM. I'm struggling to find a source but I remember hearing it in a talk given by a Googler on YouTube. I also read somewhere that moving their whole index to RAM is how they brought their response times down from a couple of seconds to a couple hundred milliseconds.
- cromwellian 5y agoHere, let Jeff Dean explain how it works (and why this is a complex problem that a lot of the naive 'build a better search engine than Google' miss) https://pdfs.semanticscholar.org/fc32/72302461b74217662085a8a05a5e500dbf05.pdf https://pdfs.semanticscholar.org/fc32/72302461b74217662085a8... The original article's description of how Google search signals work doesn't even scratch the surface of all the signals it's using.
- joe_the_user 5y agoI'd say you're half right. Things are harder now and the success of Google has contributed to this. However, I think Google has severely degenerated from just two years ago, when most of the problems were fully in effect. Google is a bit of a product of the situation of scams being the easiest way anyone makes online.
- kodah 5y ago> Nobody is coming along with a better search engine https://neeva.com/ https://neeva.com/ better than pretty much any solution (Qwant, DDG, etc) I've personally tested. It also indexes specific websites like StackOverflow, GitHub, and GMail. Edit: Neeva does require an account to create because eventually the product is going to require a subscription.
- sokoloff 5y ago"See results for '<my search term>' Create your free Neeva account." "To continue searching and access all of Neeva's features, create your free account. Already a Neeva member? Sign in" Yeah, no.
- kodah 5y agoAh, Neeva is eventually going to be a paid product, that's why it's that way. That said, their membership is pretty cheap for what you get.
- sokoloff 5y agoI don't mind giving my email or making an account after I have any inkling that the product might be worth deleting a few spam emails. Neeva was giving me a modal popup in my first minute; that's never going to get my email.
- vivekraghunatha 5y ago(Founder of Neeva here) Appreciate the input and hear your frustration. This is a carryover from changes we made to introduce the free tier earlier in the quarter. We are making imminent improvements on modal frequency and getting them out of your way. Stay posted.
- kodah 5y agoYo! Neeva is a dope product, imo, but could I offer some suggestions on how I use Neeva? 1. I have more than one GitHub account (one is for work, one is for personal stuff). I would like to somehow keep them separate, but at least being able to add multiple GitHub accounts for indexing would be cool. 2. Could Neeva offer something for segmenting the parts of my online presence? For instance, there's personal me and then there's work me. I'd like to turn a different profile on when I'm doing different things. 3. I don't know if it's possible to partner with ProtonMail, but I'd love to index that and my Proton Calendar as well. Also didn't know the free tier is permanent, but I think that's probably a good move. Thanks for what you do!
- DantesKite 5y agoI don't think these posts should be blocked, because they create discussion, which creates interest, which incentivizes for problem solving. Humans are incredibly good at solving engineering problems they can see from a mile away, although it takes time to solve.
- RC_ITR 5y agoThe progression that people almost always forget: 1) New system comes out that indexes/controls/regulates a naively created dataset 2) Data consumers adopt that system and experience benefits 3) Data suppliers learn rules of system and take advantage of it to improve the positioning of their data, thus breaking the intent of a system built on the assumption of naive creation 4) Users complain about the broken system 5) New entrants realize that the original system actually solved the core problem really well, and there are no easy ways to solve the 'gaming the system' problem 6) Flawed system remains the best available option indefinitely It's like entropy, there's just no fighting it. EDIT: And to extend this beyond Google - do you see a lot of long text blocks in 7 second tik tok videos? That's because the creator found a way to game the algorithm.
- pixl97 5y ago3) ban abusive data providers?
- elevaet 5y agoHow would that even work? Define the border between SEO and abuse.
- q-big 5y ago> Define the border between SEO and abuse. Perhaps there is no border to be defined since nearly all SEO is abuse?! The only acceptable SEO should be "provide website content that is more relevant/interesting for the viewer".
- ipsi 5y agoLet's say someone writes a blog article. They publish it on Wordpress with a bunch of unoptimised defaults. Some time after, someone comes along and says "Hey, the search engine will rank your post more highly if you include the name of the post in the URL instead of just having it be blog.example.com/posts?id=1234", so they change the URL. Is that SEO? Should that result in the site being de-listed/banned? It doesn't meet your definition of "provide website content that is more relevant/interesting for the viewer". And where is the line between "provide website content that is more relevant/interesting for the viewer" and SEO? If I realise that (to take an example from something found on Google) I could take the sentence "Identify the best customers and convert more" and rewrite it as "Marketing automation helps you identify the best customers and convert more", is that SEO because I'm intentionally adding a keyword, or is that making the content more relevant to the user by being more explicit in what I'm saying?
- Kye 5y agoPeople said this about pre-Google search engines. Someone will figure out the next PageRank and give us another 10-20 years of useful search.
- skinnymuch 5y agoThere were many multiples less people online. The internet economy was almost nothing back then. There’s no comparing how things were in the 90s and early 00s to now. Every social media company before Facebook faltered or began losing a good chunk of their user base fairly quickly. Until that stopped happening with Facebook and IG for over a decade now. Reddit’s position has also been here for a decade now.
- space_rock 5y agoSorry but duckduckgo or probably any other search is better than Google at this moment
- temp 5y agoNot everyone lives in the US and/or an English-speaking country. I can't find shit with DDG and the experience is like using Google from 20 years ago when it didn't have any of its bells and whistles.
- makeitdouble 5y agoIt’s actually because I’m not in an english speaking country that Google is failing me and screwing my searches for anything than can be somewhat pushed to local results… I’m glad it works for you, it just reminds me how different are everyone’s expectations of good search results.
- space_rock 5y agoSame here. Traveling and when I search for postgres I don't want to find the local student project as the top result
- bricemo 5y agoSome side by side comparisons just now: "pizza near me" - Google suggested a well rated place within 5 miles. DuckDuckGo suggested a pizza place 91 miles away?? "busted kids lip" - Google says gauze and a cold pack. DuckDuckGo says salt water?? "best monitor for mac" - Google says Dell Ultrasharp with 4.5 stars and 738 ratings on amazon. DuckDuckGo says BenQ 4 stars with 174 ratings on amazon? how is this better?
- Gtex555 5y agoCould you please go into more detail , I agree with you btw.
- huynhhacnguyen 5y agoI believe blocking an honest discussion is never going to do more good than harm. The original post doesn't even suggest that we should overthrow our Google overlord to replace it with whatever we find interesting. It's just pointing various limitations and issues that Google users are facing right now. And apparently from the replies here, there are even more aspects that people are have troubles with. It's true building a good search engine is an gigantic undertaking but if Google (or someone else entirely) is aware of the current issues, they may have ideas to tackle those problems and then we can all share a better internet. Same for the argument with democracy and any other similar arguments, pointing out problems within our current institutions doesn't remotely mean that we want to abolish everything and start from scratch.
- mohanmcgeek 5y ago> Google was better ten years ago because search was an easier problem to solve back then Has anyone considered the possibility that all this Machine learning and AI models is what made Google and YouTube searches worse?
- JohnFen 5y agoI strongly suspect that's a very large part of what is making everything worse. Or, at least, there does seem to be a correlation in time between the implementation of ML and the degradation of the quality of results.
- mohanmcgeek 5y agoI wouldn't call it so much the implementation of ML but rather from when they decided they were an AI/ML company. The Amazon recommendations engine, the poster child of ML in 2014 isn't any good at recommending products that I actually want to buy either.