5 ms·

The birth and death of search engine optimization

58 points by xena 3y ago

Kye 3y ago
The "wow, finally some good technology" to "wait no, it's the Torment Nexus again" pipeline.
xaedo 3y ago
[flagged]
skilled 3y ago
The dark side of SEO is a multi-billion dollar industry that no investigative journalist has ever bothered to decipher, even though it can be done in less than a year. I wish I had the capital to do it because for the last 10 years, most of what you have clicked on in Google hasn’t been earned but heavily manipulated by people who aren’t afraid to pay $400 to $5,000 dollars per backlink. Now we’re in the endgame though. Google is already rolling out features like showing follower counts in searches as a precaution against spammy sites. Who wants to click on an article for a site with 500 followers when you can click on Forbes which has millions. Need plumbing advice? Forbes got you. Need an airplane technician who lives 58 degrees North of 6th street in Austin, TX? Yep, Forbes got experts lined up for that too. Someone said it best in another thread, those who still give a shit about the web will go back to IRC and forums and that will be the end of it. You couldn’t trust Google before, now you most definitely can’t. And to think that they are rolling out all their generative features soon to compensate for OpenAI wiping the floor with them…
- whoopdedo 3y ago
  At that point why do we even need Google? Any search will lead you to the same circle of mega-sites. If only sites that are already popular get exposure that's not a search engine, that's a directory. May as well go back to "Jerry & David's Guide to the World Wide Web"
  - natebc 3y ago
    The value isn't us, that care about the web and quality search results, it's the vast majority of normals that just type stuff into Google and click away. They'll continue doing that, and not even notice, when it does turn into "Jerry & David's Guide to the World Wide Web".
- deleted 3y ago
  [deleted]
- wordpad25 3y ago
  If SEO was so easily gamed, GOOGLE competitors would’ve sidelined it by offering more relevant results. This is clearly not the case. Google heavily prioritizes users actually finding what they are looking for.
  - skilled 3y ago
    But of course the results are relevant… You wouldn’t write a novel based on the chance that someone would search for that specific type of plot. Google works the same way. It gives you results based on what you search for, but who gets to sit at the top and how did that site get to the top? Because it followed Google’s guidelines which can be offset by a single editorial backlink? That’s the angle I am talking about. There are millions of niches where big money can be made, and in those specific niches you don’t rise to the top simply because you are the best. Google has no idea what is or isn’t the best and for them it’s just as much of a guessing game. I suppose I am talking specifically about selling links. How would you feel if you found out that one of the journalists whose content you admire is also participating in these schemes? And mind you, editorial links are like the top of the food chain. Underneath that you got millions of other sites that will sell links starting anywhere from $10 to $1,000 bucks depending on their own “authority”. That’s what I would like to see investigated and exposed, because it can be done. As someone who operates a mildly successful blog, I get around 300 emails a month asking me to peddle all kinds of products and whatnot.
  - rightbyte 3y ago
    No. The competitors scrapers are banned. Only the big ones are let through. I can apply the same argument as you. "If the almighty invisible hand was allowed to compete in the search space competitors would pop up with their own indexers".
- Kye 3y ago
  There have been numerous investigations into SEO over the years. Good luck finding them, though. I'd link them if I could remember any details or find them on search engines.
- TheCleric 3y ago
  > Who wants to click on an article for a site with 500 followers when you can click on Forbes which has millions. The problem is while Forbes can have high quality articles, it also has a wide array of VERY low quality ones. https://www.niemanlab.org/2022/02/an-incomplete-history-of-forbes-com-as-a-platform-for-scams-grift-and-bad-journalism/ https://www.niemanlab.org/2022/02/an-incomplete-history-of-f... So the issue becomes it's less the domain I trust/don't trust, but rather a more complicate heuristic.
  - skilled 3y ago
    Yep. That’s the point I was making. Forbes will shovel rocks for sale if the margin is in the $5 dollar range. All these authority sites are doing it.
    - TheCleric 3y ago
      Ahh, sorry. I need more coffee!
seanwilson 3y ago
> The basic idea is that you use generative AI to systematically churn out articles about a given topic and then use A/B testing to eliminate the ones that don't "work" on search engines in favor of the ones that do. This is a way to automate the process of creating content farms. Why does PageRank not combat this? The content shouldn't rank well unless high authority sites are linking to it? And if a high authority site started linking to AI generated junk, it would start losing its authority because it would start losing backlinks. > The basic idea is that you use generative AI to systematically churn out articles about a given topic and then use A/B testing to eliminate the ones that don't "work" on search engines in favor of the ones that do. This is a way to automate the process of creating content farms. It would be good if you could select a list of sites that you trust, so the sites they link to (and the sites those link to and so on) get boosted in your own personal search results. I recall Google was trying something like this with Google+. People could +1/upvote a site, and the upvotes from the people you followed would influence personalise your search results, but this went away when Google+ was shut down.
- abound 3y ago
  > It would be good if you could select a list of sites that you trust, so the sites they link to (and the sites those link to and so on) get boosted in your own personal search results. Kagi _kinda_ supports this with "Personalized Results", you can see the most popular personalizations at [1]. To my knowledge, it doesn't do the "links to sites you trust" piece, but is still really useful. Note: I work with Kagi in a part-time role. [1] https://kagi.com/stats?stat=leaderboard https://kagi.com/stats?stat=leaderboard
- yetanother12345 3y ago
  > Why does PageRank not combat ... PageRank was the toy that got Google started, it has since been left for other toys. AFAIK around a decade ago it was succeded by the toy "quality signals". So, the reason PageRank does not is that PageRank... well, does not.
twelve40 3y ago
This article completely conflates SEO with content generation, which hasn't been the deciding factor in SEO for decades now. It also asserts that ChatGPT is better because it's not trying to sell, also completely ignoring the fact that it trains on data that also somehow has to be ranked and the fact that this ranking algorithm is hidden deeper doesn't automagically make it bulletproof to gaming.
- akavel 3y ago
  Would you mind sharing, me being not in the know, what is the deciding factor in SEO for decades, then?
  - jareklupinski 3y ago
    > what is the deciding factor in SEO for decades, then the amount of money you pay to the search provider you could literally pay Apple to insert an ad for your own app when someone searches specifically for your competitor's name
breakingrules 3y ago
[dead]
merelysounds 3y ago
> Even though yt-dlp is probably actually what you want for a tool to download video from YouTube. Is it though? My guess would be that a web app - one that doesn’t require visiting a github project page or downloading anything - would be more user friendly and more appealing for most users. As of now that seems to be the top result.
- SnaggyJoker 3y ago
  I'm fairly certain the author is speaking to the functionality of the application.
  - merelysounds 3y ago
    My point was the target audience. I.e. that an average user that enters “youtube downloader” into a search engine would not see a CLI python tool as functional for them.
troelsSteegin 3y ago
Content farms aren't new - but now they are cheaper and better. I'd think that SEO would have already made a market beneficial to "qualified" content sources - like traditional or branded media. So, a site like vox.com, which has relative to GPT chum, high quality content, asks for donations, but is nominally free. I assume they make money from ads. If a qualified publisher doen't make enough money on ads and donations/subscriptions to cover their editorial costs, and if they do generate enough ad revenue for Google, is there an incentive for Google to help fund the publisher? Is there a case where Google pays for high trust content against which to serve higher margin ads? I clearly don't know how this business works...

58 points by xena 3y ago

Kye 3y ago

The "wow, finally some good technology" to "wait no, it's the Torment Nexus again" pipeline.

xaedo 3y ago

[flagged]

skilled 3y ago

The dark side of SEO is a multi-billion dollar industry that no investigative journalist has ever bothered to decipher, even though it can be done in less than a year. I wish I had the capital to do it because for the last 10 years, most of what you have clicked on in Google hasn’t been earned but heavily manipulated by people who aren’t afraid to pay $400 to $5,000 dollars per backlink. Now we’re in the endgame though. Google is already rolling out features like showing follower counts in searches as a precaution against spammy sites. Who wants to click on an article for a site with 500 followers when you can click on Forbes which has millions. Need plumbing advice? Forbes got you. Need an airplane technician who lives 58 degrees North of 6th street in Austin, TX? Yep, Forbes got experts lined up for that too. Someone said it best in another thread, those who still give a shit about the web will go back to IRC and forums and that will be the end of it. You couldn’t trust Google before, now you most definitely can’t. And to think that they are rolling out all their generative features soon to compensate for OpenAI wiping the floor with them…

whoopdedo 3y ago
At that point why do we even need Google? Any search will lead you to the same circle of mega-sites. If only sites that are already popular get exposure that's not a search engine, that's a directory. May as well go back to "Jerry & David's Guide to the World Wide Web"
- natebc 3y ago
  The value isn't us, that care about the web and quality search results, it's the vast majority of normals that just type stuff into Google and click away. They'll continue doing that, and not even notice, when it does turn into "Jerry & David's Guide to the World Wide Web".
deleted 3y ago
[deleted]
wordpad25 3y ago
If SEO was so easily gamed, GOOGLE competitors would’ve sidelined it by offering more relevant results. This is clearly not the case. Google heavily prioritizes users actually finding what they are looking for.
- skilled 3y ago
  But of course the results are relevant… You wouldn’t write a novel based on the chance that someone would search for that specific type of plot. Google works the same way. It gives you results based on what you search for, but who gets to sit at the top and how did that site get to the top? Because it followed Google’s guidelines which can be offset by a single editorial backlink? That’s the angle I am talking about. There are millions of niches where big money can be made, and in those specific niches you don’t rise to the top simply because you are the best. Google has no idea what is or isn’t the best and for them it’s just as much of a guessing game. I suppose I am talking specifically about selling links. How would you feel if you found out that one of the journalists whose content you admire is also participating in these schemes? And mind you, editorial links are like the top of the food chain. Underneath that you got millions of other sites that will sell links starting anywhere from $10 to $1,000 bucks depending on their own “authority”. That’s what I would like to see investigated and exposed, because it can be done. As someone who operates a mildly successful blog, I get around 300 emails a month asking me to peddle all kinds of products and whatnot.
- rightbyte 3y ago
  No. The competitors scrapers are banned. Only the big ones are let through. I can apply the same argument as you. "If the almighty invisible hand was allowed to compete in the search space competitors would pop up with their own indexers".
Kye 3y ago
There have been numerous investigations into SEO over the years. Good luck finding them, though. I'd link them if I could remember any details or find them on search engines.
TheCleric 3y ago
> Who wants to click on an article for a site with 500 followers when you can click on Forbes which has millions. The problem is while Forbes can have high quality articles, it also has a wide array of VERY low quality ones. https://www.niemanlab.org/2022/02/an-incomplete-history-of-forbes-com-as-a-platform-for-scams-grift-and-bad-journalism/ https://www.niemanlab.org/2022/02/an-incomplete-history-of-f... So the issue becomes it's less the domain I trust/don't trust, but rather a more complicate heuristic.
- skilled 3y ago
  Yep. That’s the point I was making. Forbes will shovel rocks for sale if the margin is in the $5 dollar range. All these authority sites are doing it.
  - TheCleric 3y ago
    Ahh, sorry. I need more coffee!

seanwilson 3y ago

> The basic idea is that you use generative AI to systematically churn out articles about a given topic and then use A/B testing to eliminate the ones that don't "work" on search engines in favor of the ones that do. This is a way to automate the process of creating content farms. Why does PageRank not combat this? The content shouldn't rank well unless high authority sites are linking to it? And if a high authority site started linking to AI generated junk, it would start losing its authority because it would start losing backlinks. > The basic idea is that you use generative AI to systematically churn out articles about a given topic and then use A/B testing to eliminate the ones that don't "work" on search engines in favor of the ones that do. This is a way to automate the process of creating content farms. It would be good if you could select a list of sites that you trust, so the sites they link to (and the sites those link to and so on) get boosted in your own personal search results. I recall Google was trying something like this with Google+. People could +1/upvote a site, and the upvotes from the people you followed would influence personalise your search results, but this went away when Google+ was shut down.

abound 3y ago
> It would be good if you could select a list of sites that you trust, so the sites they link to (and the sites those link to and so on) get boosted in your own personal search results. Kagi _kinda_ supports this with "Personalized Results", you can see the most popular personalizations at [1]. To my knowledge, it doesn't do the "links to sites you trust" piece, but is still really useful. Note: I work with Kagi in a part-time role. [1] https://kagi.com/stats?stat=leaderboard https://kagi.com/stats?stat=leaderboard
yetanother12345 3y ago
> Why does PageRank not combat ... PageRank was the toy that got Google started, it has since been left for other toys. AFAIK around a decade ago it was succeded by the toy "quality signals". So, the reason PageRank does not is that PageRank... well, does not.

twelve40 3y ago

This article completely conflates SEO with content generation, which hasn't been the deciding factor in SEO for decades now. It also asserts that ChatGPT is better because it's not trying to sell, also completely ignoring the fact that it trains on data that also somehow has to be ranked and the fact that this ranking algorithm is hidden deeper doesn't automagically make it bulletproof to gaming.

akavel 3y ago
Would you mind sharing, me being not in the know, what is the deciding factor in SEO for decades, then?
- jareklupinski 3y ago
  > what is the deciding factor in SEO for decades, then the amount of money you pay to the search provider you could literally pay Apple to insert an ad for your own app when someone searches specifically for your competitor's name

breakingrules 3y ago

[dead]

merelysounds 3y ago

> Even though yt-dlp is probably actually what you want for a tool to download video from YouTube. Is it though? My guess would be that a web app - one that doesn’t require visiting a github project page or downloading anything - would be more user friendly and more appealing for most users. As of now that seems to be the top result.

SnaggyJoker 3y ago
I'm fairly certain the author is speaking to the functionality of the application.
- merelysounds 3y ago
  My point was the target audience. I.e. that an average user that enters “youtube downloader” into a search engine would not see a CLI python tool as functional for them.

troelsSteegin 3y ago

Content farms aren't new - but now they are cheaper and better. I'd think that SEO would have already made a market beneficial to "qualified" content sources - like traditional or branded media. So, a site like vox.com, which has relative to GPT chum, high quality content, asks for donations, but is nominally free. I assume they make money from ads. If a qualified publisher doen't make enough money on ads and donations/subscriptions to cover their editorial costs, and if they do generate enough ad revenue for Google, is there an incentive for Google to help fund the publisher? Is there a case where Google pays for high trust content against which to serve higher margin ads? I clearly don't know how this business works...