6 ms·
As the article mentions, the web has changed. It's very hard to return a list of good results when there are none. I've yet to see side-by-side comparisons wher
by xnx 3y ago
As the article mentions, the web has changed. It's very hard to return a list of good results when there are none. I've yet to see side-by-side comparisons where another search engine is doing a better job.
I've had "Search Generative Experience" (AI/LLM synthesized result/summary above the standard web results) turned on for awhile. It is definitely the future. To survive the flood of revenue optimized content (recipes being a familiar example), you need an active [software] agent working on your behalf to read content and make one straight thing out of the crooked timber of the web.
- A_D_E_P_T 3y agoThe problem isn't just that the internet has become more SEO-oriented and encumbered with low-value clickbait websites -- it's also that Google doesn't listen to instructions the way it used to. Google started slipping when it began assuming that it knows better than you do. For instance, when the exact search operator (" ") stopped working. I'd take the Google of 2014 over the Google of 2023 without thinking twice, and I think that I'm not alone in this.
- intalentive 3y agoAgree. Also their results are often biased and I sometimes find myself going to Yandex.
- selimthegrim 3y agoI think Frank Zappa would chortle with irony that a Russian search engine returns his discographies uncensored in the here and now versus an American one.
- marginalia_nu 3y agoAll search engines are inherently biased, since they perform a ranking task (which is a value judgement). They all encode (possibly uncessfully) the values of their creator. This is why having multiple search engines is important, as diversity in search options is the closes you'll get to objectivity. (said the independent search engine creator with the same air of righteousness as when Plato argued philosophers should be kings)
- midhhhthrow 3y agoYes but they’re also politically biased. Not to mention, the enormous amount of web content that is effectively banned from seArch engine results due to low prestige
- SV_BubbleTime 3y agoCongrats you are one step from re-inventing DogPile. Add AI and blockchain to pull in a ton of VC money.
- endisneigh 3y agoVerbatim search still exists. https://www.google.com/advanced_search https://www.google.com/advanced_search
- A_D_E_P_T 3y agoDo you use it often? It doesn't work the way it used to. I can't count the number of times I've searched for a technical term -- in quotes -- and got results back for a subtly (sometimes not so subtly) different term.
- endisneigh 3y agoyou have an example of this? using quotes is not the same as going to advanced search and saying use exact word or phrase. silly, yes I know.
- deleted 3y ago[deleted]
- nostrademons 3y agoQuotes is different than verbatim mode. Compare: [verbatim mode] https://www.google.com/search?q=unexpected+newline+in+composite+literal%3B+possibly+missing+comma+or+%7D&tbs=li:1 https://www.google.com/search?q=unexpected+newline+in+compos... [quotes] https://www.google.com/search?q=%22unexpected+newline+in+composite+literal%3B+possibly+missing+comma+or+%7D%22 https://www.google.com/search?q=%22unexpected+newline+in+com... The top two results are the same, but besides that verbatim mode seems way better to me.
- NegativeK 3y agoI don't even bother with quotes anymore. When I'm searching for something technical from the normal interface, I'm almost always going to Tools -> All results dropdown -> Verbatim.
- eastern 3y agoI see that I can specifically ask for results that are in the Flash .swf format. Very convenient.
- pascalxus 3y agoAbsolutely. i so often do a search and it presumes to think it knows what i want rather than giving me what I ask for, sometimes even when I'm being quite specific. I will say it does a great job for programming stuff but a lousy job whenever I'm researching economics topics.
- kjkjadksj 3y agoEven programming stuff it does a lousy job. I use R a lot. I plot with the base graphics package because it works fine. Finding R information that does not involve the tidyverse library is so damn difficult on google search. Its bests to just go straight to the documentation since stackoverflow and other sites are pretty worthless with pretty much all answers requiring ggplot or dpylr even if they can be implemented with zero dependencies with the base tooling. Same situation with python honestly. No I don’t want to use pandas, if I wanted better data wrangling I’d just wrangle in base R then import the wrangled data into python if need be instead of adding another dependency to my python code. Apparently this makes me a heretic based on google search results.
- 2OEH8eoCRo0 3y agoMore of the web is also behind paywalls or is on platforms like tiktok, instagram, twitter, etc. Less to scrape and index.
- rtsil 3y agoMy most recent complaint is the disappearance of the pagination numbers. Since I know the top results are mostly useless spams on some searches, I used to go to page 4 or 5 to find the actual results. Now to achieve the same result I have to click and scroll 5 times.
- charcircuit 3y ago>For instance, when the exact search operator (" ") stopped working. It still works.
- Zambyte 3y agoIt works, but it does something different than it used to. It used to only include results that actually contained the exact phrase you put within quotes. That is no longer the case.
- crazygringo 3y agoNow you just check "verbatim search" in the tools menu. Works exactly how quotes used to.
- charcircuit 3y agoIt works the same and it never only contained the exact phrase as Google ignores punctuation.
- unlikelytomato 3y agoThis is not my experience. I search for exact phrases the same way I always have. Three to result does not contain the word I emphasized. This never happened before a few years ago. Now it happens daily.
- charcircuit 3y agoAre you sure the page doesn't contain it? It is not a simple as just visiting the page and searching for the thing you quoted as there may be punctuation between words or the text may be in the alt text of an image or somewhere else that is not obvious.
- unlikelytomato 3y agoI am quite sure. The kinds of words in question would not even really make sense for "SEO gaming" to come into play. Typically, it presents with trying to match a word vs a phrase(I don't ever try to match phrases). I try matching smaller and smaller fragments of the word and still... nothing. I remember this working back when it was common for pages on gray websites to stuff giant blobs of text with juicy keywords out of view with janky html. Yet, somehow those pages were ranked very low and results were good. There is a clear degradation in quality. Actually, it has happened with results that include hacker news discussions. I scrape through and expand EVERY comment on a 6 month old post and still find no partial or exact matches for my search keyword. It is very clear to me that Google has switched behavior. It used to flex on how awesome it was to bring you to an esoteric corner of the internet matching your exact query. Now, it tries to converge your queries into what it thinks you want(not what you asked for) and funnel you into large websites at the expense of utility.
- bayindirh 3y agoSo, how paid search engines like Kagi manages to bring useful results every time? Its results are superior, with way less spam and SEO optimized content.
- mattnewton 3y agoI have no idea; but I speculate this is one area challengers have it easier because a) there is no giant industry optimizing for ranking on Kagi. There are whole armies of people trying to game google search, meaning whatever heuristics they have would probably be rendered useless shortly if google were to adopt them. And b) Kagi has a much smaller, more technical userbase. So like the early web they can get away with a search experience tailored for those kinds of people and not millions of people powering on a smartphone as their first computer.
- bayindirh 3y agoThe biggest power of Kagi is “you” are the algorithm. A page is optimizing for Kagi and providing bad results? Block them. You found a good website burrowed down? Raise/promote them. Kagi works for you, with your input. It doesn’t do “advanced guessing” by implicitly and opaquely processing your interaction with it.
- mattnewton 3y agoExplicit control can work for ranking but can’t scale to the retrieval step of search - you aren’t going to sift through thousands of thousands of pages of potential results to give feedback on a meaningful portion of them, there must be some method of pulling pages to be ranked. This is where the gaming happens, with basically limitless spam domains that need to be filtered out. There just is no way Kagi is 100% relying on self reports for this.
- asdff 3y agoThe whole idea of the model is to fit it on some reference so you can project it onto new data you haven't classified yet. So your first few weeks, you might get some spam sure. But eventually if you are optimizing the system by e.g. only ranking technical blogs with no seo bullshit, then technical blogs with no seo bullshit is what would fit best with that model among the new data and what you see on your results. That can't be gamed without knowing the sort of model fit you are generating and this fit is defined by the user.
- marginalia_nu 3y agoI don't think the problem, in general, is that no good results exist for any given query, more often the problem is that the good results are rarely well search engine optimized, and thus stand zero chance of making the search results. Some people admittedly expect pretty weird websites to exist. Like who would even publish a good objective comparison between products in a specific category. But people make those queries looking for those types of results, and spammers generate such content to make bucks off the traffic. 20 years ago you'd have gotten nothing in most cases.
- mgaunard 3y agoThere are a lot of people that genuinely make comparisons of specialty products.
- marginalia_nu 3y agoThat sort of stuff is typically marketing.
- mgaunard 3y agoOr simply just passionate people talking about a niche. Consider coffee. You'll find hundreds of articles comparing the most high-end grinders. You even have PhDs writing white papers on the physics of coffee and how particle size distribution achieved by different grinders affects extraction quality. Those are genuine opinions from aficionados. The same is true for all kinds of other things. The internet has a lot more than the auto-generated marketing crap you give it credit for.
- marginalia_nu 3y agoThis is true for things that are related to a hobby. There are many things where it basically doesn't exist. Nobody is writing articles comparing server colocation offerings near ${YOUR_CITY}, yet people search for them expecting them to exist.
- xnx 3y agoI'll also had that a huge portion of genuinely good content is now inaccessible to Google in walled gardens like Discord, Facebook, Twitter, and TikTok.
- SV_BubbleTime 3y agoI remember google images before Getty and Pinterest ruined it.
- hutzlibu 3y ago"To survive the flood of revenue optimized content (recipes being a familiar example), you need an active [software] agent working on your behalf to read content and make one straight thing out of the crooked timber of the web." So what do you think, will be the goal of all advertisers? To get their recommendations baked into that AI. Direct bribing/revenue sharing with the manufactur of that AI. Or sneaking data into training sets via bribed employes and so on. I don't trust black boxes and never will.
- bytefactory 3y agoPerhaps open-source AI models will be the our counterweight to commercial capture of AIs.
- asdff 3y agoThen these companies will just use closed source models.
- nine_k 3y agoRight. That's why the agent should live under your control, and receive training from your search history. Paid search is the future. Either you pay a subscription fee to something like Kagi, or you pay for hardware and electricity turn run your own "AI agent" locally.
- majormajor 3y agoI don't believe that anything Google does now - summarization included - will avoid being gamed just as effectively as their current search results have. It's a negative-sum arms race, but if Google (or whoever) wasn't spending the money to fuel the arms race by passing on ad money, who would pay for Google to exist in the first place?
- JohnFen 3y agoI am convinced that a lot of the reason why Google search went downhill is that they started using more AI. Specifically, using it to try to interpret what I'm searching for rather than taking my word for it, which leads to irrelevant search results. That's just my hypothesis, of course. I don't know for certain. But it does make me very skeptical about using AI to help search the web. I want a robust set of search modifiers instead.
- beebeepka 3y agoI liked StartPage/Ixquick results much better than what Google used to show. Maybe they still do but wouldn't know as I stopped using it after they got acquired. Didn't/don't they rely on Google?
- c7b 3y ago> To survive the flood of revenue optimized content..., you need an active [software] agent working on your behalf How long do you think it will be before that software agent's output will be just as riddled with commercialized content? I think we might have only a short window where we get to enjoy the fairly unadulterated output of the base LLMs by the large providers (yes, I'm aware that they're screening for safety already, but there could be a lot more coming). Imagine a dialogue like this: Q: What should I watch out for when buying new car tires? A current answer might be something like: Here's a list of things to watch out for when buying car tires: A, B, C, D, E. Remember to always consult with specialists. Shall I explain more about the topic? A future answer might look something like: According to experts like the professionals at [tire shop near you], here are the things you need to watch out for when buying car tires: A, B, C. You can use this coupon to get a 10% discount off your next purchase at [tire shop near you]. Shall I book an appointment for a free consultation at [tire shop near you] for you?
- autoexec 3y ago> you need an active [software] agent working on your behalf I'd love to have one, but do any exist? SGE works on google's behalf, collecting data for google who will use what they learn from it against me. I'm guessing that until we can use common hardware to train/host an AI agent ourselves, we're stuck with terrible search results unless we're willing to hand over more and more data to others who want to exploit us, while pretending we can trust them to return results that are in our best interest instead of theirs. That's not a great future to look forward to.