4 ms·

I wonder if web searches used to be pretty productive, then declined as sponsored results and SEO degraded things. Nowadays an ai assist with a web search usua

by m463 9mo ago

I wonder if web searches used to be pretty productive, then declined as sponsored results and SEO degraded things. Nowadays an ai assist with a web search usually eliminates the search altogether and gives you a clear answer right away. for example, "how much does a ford f-150 cost" will give you something ballpark in a second, compared to annoying "research" to find the answer shrouded in corporate obfuscation.

gtowey 9mo ago
I was just thinking exactly the same. Basic web search has become so horrible that AI is being used as its replacement. I found it a sad condemnation of how far the tech industry has fallen into enshittification and is failing to provide tools that are actually useful.
- gh0stcat 9mo ago
  We always had the technology to do things better, it's the money making part that has made things worse technologically speaking. In this same way, I don't see how AI will resolve the problem - our productivity was never the goal, and that won't change any time soon.
- johnnyanmac 9mo ago
  And it'll happen again when AI models start resorting to ads once again.
  - emptybits 9mo ago
    Yup. Any LLM recommendation for a product or service should be viewed with suspicion (no different than web search results or asking a commission-based human their opinion). Sponsored placements. Affiliate links. Etc. Or when asking an LLM for a comparison matrix or pros and cons between choices ... beware paid placements or sponsors. Bias could be a result of available training data (forgivable?) or due to paid prioritization (or de-prioritizing of competitors!)
    - littlecranky67 8mo ago
      Eager to see how that will work with existing laws. At least in a lot of countries in the EU, any advertisement has to be explicitly marked as such. Sponsored content, too. So the AI will have to highlight that.
  - littlecranky67 8mo ago
    I don't think that will ever happen. All you need is a trivial browser extension with a locally run, very primite LLM, that takes the output of the commercial LLM, and removes all advertisement. And adBlocker AI, so to speak. Yes, there will be people not using adblockers just as there are people today. But no adblocker ever was able to remove SEO spam from googles website, all they did was hiding obvious adds. They didn't improve the search experience.
    - johnnyanmac 8mo ago
      There will always be ad free solutions, but the most popular (or simply, most market captured tools) will be the ones most people flock to. Better services than many market leaders came along, but the leaders kept growing. Unless this bubble popping is truly catastrophic, I don't see this ending a different way.
- Wobbles42 9mo ago
  Their tools are very useful. To their customers. Not to their users.
- littlecranky67 8mo ago
  > Basic web search has become so horrible It is not horrible, it reached the point of absolute excellence. Not for you, the user - but for making money for the creator. Remember, no one paid for web search, so you are the product. If you are the provider of the web search engine, the point of having web search is not deliver the best search result to the user, but maximize the amount of money you can make from the sum of the world population. And google did very good in maximizing their profits, without users turning away from them.
monkpit 9mo ago
The turning point was around when google stopped honoring Boolean ops and quotation marks
- Wobbles42 9mo ago
  The killer app for AI might just be unenshittifying search for a couple of years. Then SEO will catch up and we'll have spam again, but now we'll be paying by the token for it. Probably right around the time hallucination drops off enough to have made this viable. I kind of want to become Amish sometimes.
  - adrianN 9mo ago
    AI automates spam generation so more than likely all hope is lost for the human driven web.
    - direwolf20 9mo ago
      Only the corporate web which values quantity over quality. Have you tried Marginalia Search? It's refreshing, although it doesn't index enough stuff to find what you're looking for, most times.
  - kakacik 9mo ago
    Too much money in ads, and search is just a huge cash pipeline straight towards it. No way we can have non-ad-infested llm search out in the wild from any major vendor in upcoming future. Google-fu just becomes llm-google-fu, while sometimes it goes off rails and then apologizes in that typical super annoying way (and screws up something else). Maybe smaller ones can somehow provide almost comparable but ad-free service, heck even mildly worse but genuine results would win many people over, this one included.
    - Wobbles42 8mo ago
      I would agree with all of that. Searching for information is inherently an adversarial process. You want your attention in one place, other actors in the system want it in another. Any solution that doesn't suck will need to be aware of this.
aurareturn 9mo ago
then declined as sponsored results and SEO degraded things It didn't decline because of this. It declined because of a general decade long trend of websites becoming paywalled and hidden behind a login. The best and most useful data is often inaccessible to crawlers. In the 2000s, everything was open because of the ad driven model. Then ad blockers, mobile subscription model, and the dominance of a few apps such as Instagram and Youtube sucking up all the ad revenue made having an open web unsustainable. How many Hacker News style open forums are left? Most open forums are dead because discussions happen on login platforms like Reddit, Facebook, Instagram, X, Discord, etc. The only reason HN is alive is because HN doesn't make need to make money. It's an ad for Y Combinator. SEO only became an issue when all there is for crawlers is SEO content instead of true genuine content.
- adrianN 9mo ago
  I seem to remember very few ads on the early web. Most sites I frequented were run by volunteers who paid out of their own pockets for webspace.
  - aurareturn 8mo ago
    What year? 90s? I remember ads everywhere in 2000s.
- DrScientist 8mo ago
  > The best and most useful data is often inaccessible to crawlers. Interesting point. > ost open forums are dead because discussions happen on login platforms like Reddit, Facebook, Instagram, X, Discord, etc Ironically isn't one of the reasons some of those platforms started to use logins was so they could track users and better sell their information to ad people? Obviously now there are other reasons as well - regulation, age verification etc. Does this suggest that the AI/ad platforms need to tweak their economic model to share more of the revenue with content creators?
- badc0ffee 8mo ago
  You can still use Reddit without logging in. In fact it's completely unlike Discord. Lots of Reddit discussions still show up in web search results.
kdheiwns 9mo ago
I used to be able to google a question like that and get an accurate answer within the top 3 results nearly every time about 20 years ago. Then it got worse and worse and became pretty much completely useless about 10 years ago. Now AI will give me a confident answer that is outright wrong 20% of the time or kind of right but not really 30% of the time. So now I ask something using an AI chatbot and carefully word it so as to have it not get off topic and focus on what I actually want to know, wait 30 seconds for its long ass answer to finish, skim it for the relevant parts, then google the answer and try to see where the AI sourced its answer from and determine whether it misinterpreted/mixed up results or it's accurate. What used to be a 10 second google search is now a 2-3 minute exercise. I can see very much how people say AI has somehow led to productivity losses. It's shit like this, and it floods the internet and makes real info harder to find, making this cycle worse and worse and take more and more time for basic stuff.
- direwolf20 9mo ago
  Now Google has an AI answer at the top with links to sources. This streamlines the process.
- cons0le 8mo ago
  Web scraping for LLMs has almost completely ruined the search experience. In the past I could search for simple questions, and quickly get an answer without even having to click through to the links. This was horrible for web traffic, but the utility level was off the charts. It was possible to get accurate results in milliseconds. It was faster than using an LLM. Now sites put almost no info in the search result headers, to get people to click through. I think this will work on some users, but most will start using LLMs as search by default. Search engines have gotten so bad that I almost feel forced to try running SearXNG or some other search engine locally. Its a pain to set up, but degooglefication is always worth it.
  - plagiarist 8mo ago
    Is SearXNG a nice experience? I need something that specifically excludes AI slop affiliate marketing listicles. I don't know if content other than that exists out there any more. Still, I'd rather have no answer than a "top ten $item" list just describing the first page of Amazon results.
fifilura 9mo ago
My mother lost her phone so I asked her to search for "find my iphone" on Google. The result started with 3 "sponsored links" which threw her down the rabbit hole. This used to be easy.
kranner 8mo ago
Allegedly the 'clear' answer is much easier to manipulate than gaming PageRank ever was: https://x.com/thomasgermain/status/2024165514155536746 https://x.com/thomasgermain/status/2024165514155536746
- littlecranky67 8mo ago
  Don't think that is a fair point, the manipulation was done on a topic of which there are hardly any other sources (hot dog eating competition winner). If you want to manipulate what an AI tells you is the F-150 street price, you will complete with hundreds of sources. The AI will unlikely pick yours.
  - luke5441 8mo ago
    The marketing game is already moving to game LLMs. Somehow you have to get what you want to have into the training data or the context window. Currently it is probably just mostly quantity that does the trick w.r.t. training data. So e.g. spam the Internet with "product comparisons" featuring your product as the winner.
    - hombre_fatal 8mo ago
      Shifting the balance on training data seems like the wrong approach vs focusing on showing up in agent search tool results and swaying them there. It’s been a long time since agents couldn’t even conduct web search and could only riff off their model. But the examples in this thread are things an agent would search for immediately, and agents are leaning harder on tool calls and external info over time, not less.
keybored 8mo ago
> I wonder if web searches used to be pretty productive, then declined as sponsored results and SEO degraded things. Used to be. > Nowadays an ai assist with a web search usually eliminates the search altogether and gives you a clear answer right away. Now.

by m463 9mo ago

gtowey 9mo ago

I was just thinking exactly the same. Basic web search has become so horrible that AI is being used as its replacement. I found it a sad condemnation of how far the tech industry has fallen into enshittification and is failing to provide tools that are actually useful.

gh0stcat 9mo ago
We always had the technology to do things better, it's the money making part that has made things worse technologically speaking. In this same way, I don't see how AI will resolve the problem - our productivity was never the goal, and that won't change any time soon.
johnnyanmac 9mo ago
And it'll happen again when AI models start resorting to ads once again.
- emptybits 9mo ago
  Yup. Any LLM recommendation for a product or service should be viewed with suspicion (no different than web search results or asking a commission-based human their opinion). Sponsored placements. Affiliate links. Etc. Or when asking an LLM for a comparison matrix or pros and cons between choices ... beware paid placements or sponsors. Bias could be a result of available training data (forgivable?) or due to paid prioritization (or de-prioritizing of competitors!)
  - littlecranky67 8mo ago
    Eager to see how that will work with existing laws. At least in a lot of countries in the EU, any advertisement has to be explicitly marked as such. Sponsored content, too. So the AI will have to highlight that.
- littlecranky67 8mo ago
  I don't think that will ever happen. All you need is a trivial browser extension with a locally run, very primite LLM, that takes the output of the commercial LLM, and removes all advertisement. And adBlocker AI, so to speak. Yes, there will be people not using adblockers just as there are people today. But no adblocker ever was able to remove SEO spam from googles website, all they did was hiding obvious adds. They didn't improve the search experience.
  - johnnyanmac 8mo ago
    There will always be ad free solutions, but the most popular (or simply, most market captured tools) will be the ones most people flock to. Better services than many market leaders came along, but the leaders kept growing. Unless this bubble popping is truly catastrophic, I don't see this ending a different way.
Wobbles42 9mo ago
Their tools are very useful. To their customers. Not to their users.
littlecranky67 8mo ago
> Basic web search has become so horrible It is not horrible, it reached the point of absolute excellence. Not for you, the user - but for making money for the creator. Remember, no one paid for web search, so you are the product. If you are the provider of the web search engine, the point of having web search is not deliver the best search result to the user, but maximize the amount of money you can make from the sum of the world population. And google did very good in maximizing their profits, without users turning away from them.

monkpit 9mo ago

The turning point was around when google stopped honoring Boolean ops and quotation marks

Wobbles42 9mo ago
The killer app for AI might just be unenshittifying search for a couple of years. Then SEO will catch up and we'll have spam again, but now we'll be paying by the token for it. Probably right around the time hallucination drops off enough to have made this viable. I kind of want to become Amish sometimes.
- adrianN 9mo ago
  AI automates spam generation so more than likely all hope is lost for the human driven web.
  - direwolf20 9mo ago
    Only the corporate web which values quantity over quality. Have you tried Marginalia Search? It's refreshing, although it doesn't index enough stuff to find what you're looking for, most times.
- kakacik 9mo ago
  Too much money in ads, and search is just a huge cash pipeline straight towards it. No way we can have non-ad-infested llm search out in the wild from any major vendor in upcoming future. Google-fu just becomes llm-google-fu, while sometimes it goes off rails and then apologizes in that typical super annoying way (and screws up something else). Maybe smaller ones can somehow provide almost comparable but ad-free service, heck even mildly worse but genuine results would win many people over, this one included.
  - Wobbles42 8mo ago
    I would agree with all of that. Searching for information is inherently an adversarial process. You want your attention in one place, other actors in the system want it in another. Any solution that doesn't suck will need to be aware of this.

aurareturn 9mo ago

then declined as sponsored results and SEO degraded things It didn't decline because of this. It declined because of a general decade long trend of websites becoming paywalled and hidden behind a login. The best and most useful data is often inaccessible to crawlers. In the 2000s, everything was open because of the ad driven model. Then ad blockers, mobile subscription model, and the dominance of a few apps such as Instagram and Youtube sucking up all the ad revenue made having an open web unsustainable. How many Hacker News style open forums are left? Most open forums are dead because discussions happen on login platforms like Reddit, Facebook, Instagram, X, Discord, etc. The only reason HN is alive is because HN doesn't make need to make money. It's an ad for Y Combinator. SEO only became an issue when all there is for crawlers is SEO content instead of true genuine content.

adrianN 9mo ago
I seem to remember very few ads on the early web. Most sites I frequented were run by volunteers who paid out of their own pockets for webspace.
- aurareturn 8mo ago
  What year? 90s? I remember ads everywhere in 2000s.
DrScientist 8mo ago
> The best and most useful data is often inaccessible to crawlers. Interesting point. > ost open forums are dead because discussions happen on login platforms like Reddit, Facebook, Instagram, X, Discord, etc Ironically isn't one of the reasons some of those platforms started to use logins was so they could track users and better sell their information to ad people? Obviously now there are other reasons as well - regulation, age verification etc. Does this suggest that the AI/ad platforms need to tweak their economic model to share more of the revenue with content creators?
badc0ffee 8mo ago
You can still use Reddit without logging in. In fact it's completely unlike Discord. Lots of Reddit discussions still show up in web search results.

kdheiwns 9mo ago

I used to be able to google a question like that and get an accurate answer within the top 3 results nearly every time about 20 years ago. Then it got worse and worse and became pretty much completely useless about 10 years ago. Now AI will give me a confident answer that is outright wrong 20% of the time or kind of right but not really 30% of the time. So now I ask something using an AI chatbot and carefully word it so as to have it not get off topic and focus on what I actually want to know, wait 30 seconds for its long ass answer to finish, skim it for the relevant parts, then google the answer and try to see where the AI sourced its answer from and determine whether it misinterpreted/mixed up results or it's accurate. What used to be a 10 second google search is now a 2-3 minute exercise. I can see very much how people say AI has somehow led to productivity losses. It's shit like this, and it floods the internet and makes real info harder to find, making this cycle worse and worse and take more and more time for basic stuff.

direwolf20 9mo ago
Now Google has an AI answer at the top with links to sources. This streamlines the process.
cons0le 8mo ago
Web scraping for LLMs has almost completely ruined the search experience. In the past I could search for simple questions, and quickly get an answer without even having to click through to the links. This was horrible for web traffic, but the utility level was off the charts. It was possible to get accurate results in milliseconds. It was faster than using an LLM. Now sites put almost no info in the search result headers, to get people to click through. I think this will work on some users, but most will start using LLMs as search by default. Search engines have gotten so bad that I almost feel forced to try running SearXNG or some other search engine locally. Its a pain to set up, but degooglefication is always worth it.
- plagiarist 8mo ago
  Is SearXNG a nice experience? I need something that specifically excludes AI slop affiliate marketing listicles. I don't know if content other than that exists out there any more. Still, I'd rather have no answer than a "top ten $item" list just describing the first page of Amazon results.

fifilura 9mo ago

My mother lost her phone so I asked her to search for "find my iphone" on Google. The result started with 3 "sponsored links" which threw her down the rabbit hole. This used to be easy.

kranner 8mo ago

Allegedly the 'clear' answer is much easier to manipulate than gaming PageRank ever was: https://x.com/thomasgermain/status/2024165514155536746 https://x.com/thomasgermain/status/2024165514155536746

littlecranky67 8mo ago
Don't think that is a fair point, the manipulation was done on a topic of which there are hardly any other sources (hot dog eating competition winner). If you want to manipulate what an AI tells you is the F-150 street price, you will complete with hundreds of sources. The AI will unlikely pick yours.
- luke5441 8mo ago
  The marketing game is already moving to game LLMs. Somehow you have to get what you want to have into the training data or the context window. Currently it is probably just mostly quantity that does the trick w.r.t. training data. So e.g. spam the Internet with "product comparisons" featuring your product as the winner.
  - hombre_fatal 8mo ago
    Shifting the balance on training data seems like the wrong approach vs focusing on showing up in agent search tool results and swaying them there. It’s been a long time since agents couldn’t even conduct web search and could only riff off their model. But the examples in this thread are things an agent would search for immediately, and agents are leaning harder on tool calls and external info over time, not less.

keybored 8mo ago

> I wonder if web searches used to be pretty productive, then declined as sponsored results and SEO degraded things. Used to be. > Nowadays an ai assist with a web search usually eliminates the search altogether and gives you a clear answer right away. Now.