5 ms·

I’m not precisely sure what problem the author is talking about. Is it the fact that some sites have built a business model around search results or is it that

by efitz 2y ago

I’m not precisely sure what problem the author is talking about. Is it the fact that some sites have built a business model around search results or is it that Google changed it search algorithm and they don’t like the way they are prioritized or is it something else? It seems kind of unreasonable to expect Google to never experiment with their algorithm; and unfortunately at its core it is a zero sum game. You might be a winner today but a loser tomorrow. if your concern is about revenue, sharing or referrals or ad placements or ??? then I would point out that it’s very unwise to build a business whose success is based entirely on the whims of another business. I think search in general is becoming a very poor way to discover content as it is slowly getting planted by LLMs and also for years has been gamified by SEO. I think that the right model for content discovery is either crowd sourced by a like-minded community, like hacker news or curated; if the curator or community drifts away from your interests, then you have to find a new one, but oddly enough, this can actually be done within the same framework.

23B1 2y ago
> it’s very unwise to build a business whose success is based entirely on the whims of another business. Show me a business free of dependencies and I'll eat my hat.
- no_wizard 2y ago
  Everything exists in a system. I suppose the least (and therefore most resistant) to dependency related issues is small family farmers that run things like U Pick farms and grow a diversity of crops
- RiverCrochet 2y ago
  Bad argument. Having all of ones business dependencies to be that from a single business, subject to its whims, is unwise regardless of the fact that no business can exist without dependencies at all.
  - 23B1 2y ago
    Yes, full redundancy in one's entire supply chain, corporate structure, business services, infra, and finances is both efficient and feasible. 9_9
refulgentis 2y ago
In a sentence, the theory is that as search added optimization layer based on ML, after optimization layer based on ML, it's nigh-debuggable. Put another way, Google launched 1000 experiments that got +0.2% CTR and seemed innocuous, and now they have a system that only wants to give out Reddit and Quora links. I've seen this story on Google's private Blind section multiple times. Usually, coupled to discussion about a more cautionary approach taken by the pre-2020 head of search, who was worried about this outcome. IMHO they're feeling out the same outcome without the benefit of the above knowledge. I don't see this theory well-understood outside Googlers, modulo a pair of popular blog posts, whose name I can't recall, that hit the nail on the head. (disclaimer: xoogler as of oct 2023, didn't work on search)
- Filligree 2y ago
  I agree. But it's a tough problem, isn't it? I don't just mean filtering out the spam; that's a hard problem on its own. But the good-click metrics... if serving up reddit and quora links at #1 makes more people satisfied with the first result, defining satisfied as 'reads the #1 page and doesn't go back for more', then... What's wrong with that metric? That isn't a rhetorical question. It's tempting to claim the users are wrong to prefer it, but not very nice. Why do they do so?
  - refulgentis 2y ago
    Users aren't wrong (tangentially, I strongly believe they don't like it) Bravely flattening a root cause analysis to one agreeable-sounding villain, I'd do Goodhart's Law. Click-through rate not being a great proxy for user satisfaction. i.e. I always clicked reddit first, I used to append reddit to my searches. But I get a very uneasy feeling now that everything has reddit 3 times in top ten. Impossible. It's a big world. It'd be like if Encyclopedia Britannica started just including the snippet-sized version of Wikipedia articles. Not what I'm looking for from that product.
  - throwawayffffas 2y ago
    > if serving up reddit and quora links at #1 makes more people satisfied with the first result. It does not, what google is optimizing for is ad click-through not organic results click-through. The whole model is you will search for X see a couple of ads and a sea of bad results and opt to click on the ad. It's the same reason that if you search for "graphics card" on amazon and sort by price you will get 20 pages of "Graphics card Holders" and "Graphics Card Elbows". They want you to not actually use the search just click on their ad.
dangus 2y ago
Also the article did a terrible job by confusing LLM and AI query results with Google’s not-LLM search engine. And yes, like you alluded to, search engines aren’t the only way to acquire your readership or regular customers. Some media outlets basically don’t even use search engines to drive traffic, like so many social media based businesses out there. There are hundreds of thousands of businesses that barely have a linktree page.
rkagerer 2y ago
Speaking of which, I would love a curated Android app store. It says a lot that you still can't filter by ad-free apps in it.
- colordrops 2y ago
  F-droid and Aurora
  - nine_k 2y ago
    F-droid is good, but where's the curation part?
    - colordrops 2y ago
      What type of curation are you looking for? F-droid curates for open source and "anti-features" which are documented on each app page. And Aurora allows you to search Google Play while filtering out apps that utilize ads. What exactly do you want?
    - cyberax 2y ago
      It's curated. It clearly marks "anti-features" of the apps.
- Ferret7446 2y ago
  You can help create one, since unlike iOS, Android doesn't have an app market monopoly. There are already many alternative app stores.
- mrandish 2y ago
  App Finder in the Play Store will filter by ad-free and a bunch of other parameters.
raincole 2y ago
> I think that the right model for content discovery is either crowd sourced by a like-minded community, like hacker news or curated This is how all the subreddits that are remotely related to politics or news got so biased. Of course I don't know what the alternative is. If I did I would be making that ideal site instead of scrolling HN.
nox101 2y ago
> I think that the right model for content discovery is either crowd sourced by a like-minded community, like hacker news or curated; I'm curious how you'd scale that. Google gets 8.5 billion searches per day. 8.5 billion "Ask HN: Where's the best resource for X" isn't going to scale to that.
- onionisafruit 2y ago
  I may be wrong, but I don’t think op is responsible for scaling their suggestion up to serve everybody on the internet.
  - nine_k 2y ago
    Then it's a model for discovery of a tiny sliver of content :shrug: Google is now moving towards the position AltaVista used to have: everybody is using this search engine because of its breadth of coverage, the users are often unhappy with having to search for a long time and adjust the query many times, and complain about the inefficiency of it. I hope Kagi will grow and attain a large enough market share to afford indexing more of the web. Maybe we'll have a really useful search engine again, like it's 2001. For a few more years at least :-/
- cyberax 2y ago
  > Where's the best resource for X" Reddit is a pretty credible alternative for that.
  - nox101 2y ago
    That is not my experience. The mods there close any question that's been asked before even if the previous one is out of date and is closed to new answers. They also just seem to close most questions period.
    - romanows 2y ago
      Are you sure you're not thinking about Stack Overflow? I hate defending reddit ever since the API debacle, but there are scads of subreddits, many with different sets of mods, and all of the ones that help me with product reviews don't seem to have aggressive moderation? Edit: for example, I was searching for woodworking design software this morning and got a lot of helpful-looking results from reddit.
dghlsakjg 2y ago
> then I would point out that it’s very unwise to build a business whose success is based entirely on the whims of another business. When that other business is a monopoly, what choice do you have? The rise of google has effectively killed all other sources of traffic. Web sites used to get their traffic from things like webrings, directories and a variety of smaller search engines. Now? Google, or one of its properties, and to a smaller degree Meta are basically it. The curator and community discovery model is a victim of google, not a solution to it.
- dismalaf 2y ago
  Reddit, Instagram, Facebook, Tik Tok, even YouTube (Google property but not Google the search engine) are all paths to discovery. To a lesser degree sites like HN. There's also nothing actually stopping anyone from going to Bing or something. My Chrome browser takes me to Bing if I type in it or change the search engine... Also de-ranking SEO spammers is a net positive IMO.
  - mandeepj 2y ago
    > Reddit, Instagram, Facebook, Tik Tok, even YouTube (Google property but not Google the search engine) are all paths to discovery. Add blogs, News sites, and forums - to that as well.
  - dghlsakjg 2y ago
    They are paths to discovery, but are they actual drivers of traffic? My impression is that if you aren’t listed on google, your site is pretty close to dead. In any case, if you somehow managed to drive traffic via other social media, you are one underpaid moderator’s hasty decision from ruin. The point isn’t about deranking spammers it is about concentrating a huge majority of traffic (and therefore power) in the hands of a single entity which is unaccountable to anyone.
    - tempest_ 2y ago
      Another thing to consider is that those sites would rather you did you click those links. They don't benefit when you leave the platform and they down rank things they think will cause users to leave the walled garden regardless of the content.
thayne 2y ago
> it is slowly getting planted by LLMs and also for years has been gamified by SEO. I think that is basically what they are complaining about, from a content creator's perspective. They are upset that their quality content (according to them at least) is losing to ML generated garbage, and that google engineers don't even seem to be able to understand why that happens.
- lupire 2y ago
  Maybe the Google Search engineers should ask the team next door, that has been receiving the lion's share of Google's funding over the last 3 years years, with the express goal of doing exactly that?
emmelaich 2y ago
It's a problem that rankings changed so dramatically and no-one really understands why. Perhaps the ranking was terrible before and now is better. Or is it the other way around?
RA2lover 2y ago
The capability of A/B testing should have made this experimentation a one-armed bandit problem, which is far more ammenable.
fallous 2y ago
"I think that the right model for content discovery is either crowd sourced by a like-minded community" and now you've re-created the original Yahoo search, which was killed off in the original Search Engine Wars by the likes of AltaVista and later Google because it didn't take long for the amount of new content on the web to overwhelm the manual curation process. A manually curated system can only truly work with a constrained domain of content/subject matter and tolerance for high lag regarding new information. Wikipedia is an example of what that kind of system would look like.
baranul 2y ago
My take on what the author was getting at, is how Google's manipulation of its algorithm has led to poor results and backfired on them. Not just for the users of its search engine, but small businesses counting on being discovered that way. Arguably, the problem is black-box algorithms and how they are manipulated. The same problem can be faced by any curated community, where there is little to no transparency. Curated can still mean manipulated. People can jump from place to place, but that may not be advantageous, because where they want to be or the most likely places for discovery are not available.
aspenmayer 2y ago
> I think search in general is becoming a very poor way to discover content as it is slowly getting planted by LLMs and also for years has been gamified by SEO. Did you mean supplanted here?
jumping_frog 2y ago
I think you should read the blogpost again. Google wants these smaller websites to come into search results but it can't coax the The Algorithm to do it. That's why they are actively trying to debug these issues.

by efitz 2y ago

23B1 2y ago

> it’s very unwise to build a business whose success is based entirely on the whims of another business. Show me a business free of dependencies and I'll eat my hat.

no_wizard 2y ago
Everything exists in a system. I suppose the least (and therefore most resistant) to dependency related issues is small family farmers that run things like U Pick farms and grow a diversity of crops
RiverCrochet 2y ago
Bad argument. Having all of ones business dependencies to be that from a single business, subject to its whims, is unwise regardless of the fact that no business can exist without dependencies at all.
- 23B1 2y ago
  Yes, full redundancy in one's entire supply chain, corporate structure, business services, infra, and finances is both efficient and feasible. 9_9

refulgentis 2y ago

In a sentence, the theory is that as search added optimization layer based on ML, after optimization layer based on ML, it's nigh-debuggable. Put another way, Google launched 1000 experiments that got +0.2% CTR and seemed innocuous, and now they have a system that only wants to give out Reddit and Quora links. I've seen this story on Google's private Blind section multiple times. Usually, coupled to discussion about a more cautionary approach taken by the pre-2020 head of search, who was worried about this outcome. IMHO they're feeling out the same outcome without the benefit of the above knowledge. I don't see this theory well-understood outside Googlers, modulo a pair of popular blog posts, whose name I can't recall, that hit the nail on the head. (disclaimer: xoogler as of oct 2023, didn't work on search)

Filligree 2y ago
I agree. But it's a tough problem, isn't it? I don't just mean filtering out the spam; that's a hard problem on its own. But the good-click metrics... if serving up reddit and quora links at #1 makes more people satisfied with the first result, defining satisfied as 'reads the #1 page and doesn't go back for more', then... What's wrong with that metric? That isn't a rhetorical question. It's tempting to claim the users are wrong to prefer it, but not very nice. Why do they do so?
- refulgentis 2y ago
  Users aren't wrong (tangentially, I strongly believe they don't like it) Bravely flattening a root cause analysis to one agreeable-sounding villain, I'd do Goodhart's Law. Click-through rate not being a great proxy for user satisfaction. i.e. I always clicked reddit first, I used to append reddit to my searches. But I get a very uneasy feeling now that everything has reddit 3 times in top ten. Impossible. It's a big world. It'd be like if Encyclopedia Britannica started just including the snippet-sized version of Wikipedia articles. Not what I'm looking for from that product.
- throwawayffffas 2y ago
  > if serving up reddit and quora links at #1 makes more people satisfied with the first result. It does not, what google is optimizing for is ad click-through not organic results click-through. The whole model is you will search for X see a couple of ads and a sea of bad results and opt to click on the ad. It's the same reason that if you search for "graphics card" on amazon and sort by price you will get 20 pages of "Graphics card Holders" and "Graphics Card Elbows". They want you to not actually use the search just click on their ad.

dangus 2y ago

Also the article did a terrible job by confusing LLM and AI query results with Google’s not-LLM search engine. And yes, like you alluded to, search engines aren’t the only way to acquire your readership or regular customers. Some media outlets basically don’t even use search engines to drive traffic, like so many social media based businesses out there. There are hundreds of thousands of businesses that barely have a linktree page.

rkagerer 2y ago

Speaking of which, I would love a curated Android app store. It says a lot that you still can't filter by ad-free apps in it.

colordrops 2y ago
F-droid and Aurora
- nine_k 2y ago
  F-droid is good, but where's the curation part?
  - colordrops 2y ago
    What type of curation are you looking for? F-droid curates for open source and "anti-features" which are documented on each app page. And Aurora allows you to search Google Play while filtering out apps that utilize ads. What exactly do you want?
  - cyberax 2y ago
    It's curated. It clearly marks "anti-features" of the apps.
Ferret7446 2y ago
You can help create one, since unlike iOS, Android doesn't have an app market monopoly. There are already many alternative app stores.
mrandish 2y ago
App Finder in the Play Store will filter by ad-free and a bunch of other parameters.

raincole 2y ago

> I think that the right model for content discovery is either crowd sourced by a like-minded community, like hacker news or curated This is how all the subreddits that are remotely related to politics or news got so biased. Of course I don't know what the alternative is. If I did I would be making that ideal site instead of scrolling HN.

nox101 2y ago

> I think that the right model for content discovery is either crowd sourced by a like-minded community, like hacker news or curated; I'm curious how you'd scale that. Google gets 8.5 billion searches per day. 8.5 billion "Ask HN: Where's the best resource for X" isn't going to scale to that.

onionisafruit 2y ago
I may be wrong, but I don’t think op is responsible for scaling their suggestion up to serve everybody on the internet.
- nine_k 2y ago
  Then it's a model for discovery of a tiny sliver of content :shrug: Google is now moving towards the position AltaVista used to have: everybody is using this search engine because of its breadth of coverage, the users are often unhappy with having to search for a long time and adjust the query many times, and complain about the inefficiency of it. I hope Kagi will grow and attain a large enough market share to afford indexing more of the web. Maybe we'll have a really useful search engine again, like it's 2001. For a few more years at least :-/
cyberax 2y ago
> Where's the best resource for X" Reddit is a pretty credible alternative for that.
- nox101 2y ago
  That is not my experience. The mods there close any question that's been asked before even if the previous one is out of date and is closed to new answers. They also just seem to close most questions period.
  - romanows 2y ago
    Are you sure you're not thinking about Stack Overflow? I hate defending reddit ever since the API debacle, but there are scads of subreddits, many with different sets of mods, and all of the ones that help me with product reviews don't seem to have aggressive moderation? Edit: for example, I was searching for woodworking design software this morning and got a lot of helpful-looking results from reddit.

dghlsakjg 2y ago

> then I would point out that it’s very unwise to build a business whose success is based entirely on the whims of another business. When that other business is a monopoly, what choice do you have? The rise of google has effectively killed all other sources of traffic. Web sites used to get their traffic from things like webrings, directories and a variety of smaller search engines. Now? Google, or one of its properties, and to a smaller degree Meta are basically it. The curator and community discovery model is a victim of google, not a solution to it.

dismalaf 2y ago
Reddit, Instagram, Facebook, Tik Tok, even YouTube (Google property but not Google the search engine) are all paths to discovery. To a lesser degree sites like HN. There's also nothing actually stopping anyone from going to Bing or something. My Chrome browser takes me to Bing if I type in it or change the search engine... Also de-ranking SEO spammers is a net positive IMO.
- mandeepj 2y ago
  > Reddit, Instagram, Facebook, Tik Tok, even YouTube (Google property but not Google the search engine) are all paths to discovery. Add blogs, News sites, and forums - to that as well.
- dghlsakjg 2y ago
  They are paths to discovery, but are they actual drivers of traffic? My impression is that if you aren’t listed on google, your site is pretty close to dead. In any case, if you somehow managed to drive traffic via other social media, you are one underpaid moderator’s hasty decision from ruin. The point isn’t about deranking spammers it is about concentrating a huge majority of traffic (and therefore power) in the hands of a single entity which is unaccountable to anyone.
  - tempest_ 2y ago
    Another thing to consider is that those sites would rather you did you click those links. They don't benefit when you leave the platform and they down rank things they think will cause users to leave the walled garden regardless of the content.

thayne 2y ago

> it is slowly getting planted by LLMs and also for years has been gamified by SEO. I think that is basically what they are complaining about, from a content creator's perspective. They are upset that their quality content (according to them at least) is losing to ML generated garbage, and that google engineers don't even seem to be able to understand why that happens.

lupire 2y ago
Maybe the Google Search engineers should ask the team next door, that has been receiving the lion's share of Google's funding over the last 3 years years, with the express goal of doing exactly that?

emmelaich 2y ago

It's a problem that rankings changed so dramatically and no-one really understands why. Perhaps the ranking was terrible before and now is better. Or is it the other way around?

RA2lover 2y ago

The capability of A/B testing should have made this experimentation a one-armed bandit problem, which is far more ammenable.

fallous 2y ago

"I think that the right model for content discovery is either crowd sourced by a like-minded community" and now you've re-created the original Yahoo search, which was killed off in the original Search Engine Wars by the likes of AltaVista and later Google because it didn't take long for the amount of new content on the web to overwhelm the manual curation process. A manually curated system can only truly work with a constrained domain of content/subject matter and tolerance for high lag regarding new information. Wikipedia is an example of what that kind of system would look like.

baranul 2y ago

My take on what the author was getting at, is how Google's manipulation of its algorithm has led to poor results and backfired on them. Not just for the users of its search engine, but small businesses counting on being discovered that way. Arguably, the problem is black-box algorithms and how they are manipulated. The same problem can be faced by any curated community, where there is little to no transparency. Curated can still mean manipulated. People can jump from place to place, but that may not be advantageous, because where they want to be or the most likely places for discovery are not available.

aspenmayer 2y ago

> I think search in general is becoming a very poor way to discover content as it is slowly getting planted by LLMs and also for years has been gamified by SEO. Did you mean supplanted here?

jumping_frog 2y ago

I think you should read the blogpost again. Google wants these smaller websites to come into search results but it can't coax the The Algorithm to do it. That's why they are actively trying to debug these issues.