5 ms·

Local business owners I've talk to who have splurged with an SEO marketing firm this decade regret it almost across the board. After spending many hours adding

Local business owners I've talk to who have splurged with an SEO marketing firm this decade regret it almost across the board. After spending many hours adding blog posts and videography to their website to impress the google bot (but which add ~0 value to real clients) their conversions didn't improve noticeably over the simple business card website they'd had before. And the clients they do get are much more trouble than the referrals they get through satisfied clients. SEO is no longer influential over the kinds of clients you want.

itronitron 2y ago
I'm not sure whether it's SEO or AI that is to blame but in the past few years it seems like almost every article on the internet repeats itself in some form at least three times. Most blog posts about a topic feel like they were written as a High School English project. It's actually quite astonishing how bad the internet has become at conveying information compared to what it could be doing. So I'm tossing around an idea in my head, of implementing a search engine that computes a 'redundancy penalty' on every web page it indexed. The measure of redundancy would need to be multi-scale and consider repeated phrases, points made, and paragraphs as well, so not necessarily a simple problem. If you think this is something worth pursuing just let me know by commenting below.
- elevation 2y ago
  Three thoughts: First, effective writing will almost always restate the main idea somehow; you'll want your algorithm to be more sophisticated than "demote content that seems to repeat itself." Second, your novel algorithm will be expensive to compute, so you'll need some other signals to filter out very low value pages before you pay to analyze others. Finally, your technique will be valuable as a ranking signal until you (or someone else) begins to use your algorithm to help steer non trivial purchasing decisions. At that point, someone will dress up their low-value (or negative value) content to appear "high value" to your algorithm and your output will be filled with spam. You can delay this effect by keeping your user-base small, so consider selling private access to institutions who want a unique view of the internet for research or intelligence purposes. Good luck!
  - Nextgrid 2y ago
    > someone will dress up their low-value content to appear "high value" to your algorithm You are falling into the trap of thinking that Google and its competitors are victims of this. They are not. They are knowingly letting themselves get "exploited" because it turns out that those spam sites contain ads (that may be Google's) or analytics (that may be Google's), or make the search result page ads (from Google) look good in comparison and more likely to be clicked. The heuristic to detect and block spam in general is very simple. Spam is there to get you to either buy something (the signal is the presence of a credit card form, "buy" call to action, or links that ultimately lead to the former) or view ads (signal there is presence of ads). This would be very hard to game because those signals only work when they are visible (and obnoxious), which also makes them trivially detectable purely at the visual level by a crude classifier (trained on website screenshots with ads vs the same page with an ad blocker). The resulting penalty score can be used as a ranking signal, so that all other signals being equal, a result without ads would rank higher than the one with ads for a given query. This is not rocket science. The problem is that the mainstream search engines have the same business model as the spammers (and profit off each other), so there is no reason to suddenly slaughter the cash cow. The myth that search engines are "victims" to spammers/SEO provides very convenient plausible deniability so they have no reason to disprove that either.
    - Mindwipe 2y ago
      This scoring is awful. Literally what you have done here is incentivise sites from well funded bad actors, or sites that push more elaborate scams on elderly people. A plausible and sustainable funding model is a good sign for sites.
      - Nextgrid 2y ago
        It's about incentivizing sites to do what the user wants them to do. As a user, 99% of the time I am not out to buy something, so all sites that do try to sell me something are just wasting both my time and their server resources. This isn't a complete blanket ban on ads or the commercial web. If there isn't a profitable way to run a website that provides the content matching the search query, the ad-infested website will still come up first. But now, a hobbyist, non-profit or even commercial enterprise that doesn't directly sell to you has a chance to outrank the ad-infested garbage.
- thwarted 2y ago
  > almost every article on the internet repeats itself in some form at least three times Even videos like YouTube shorts, which are meant to be short, have this problem. They are seriously padded with repetition. Here's thing, have you seen this before? Few people have seen this. Do you know how it works? I'm going to show you. People are surprised when they find out how it works. But I explain it. There must not be stats on choosing "never suggest this channel again", I select that pretty consistently when I'm looking for short form content and these videos waste all that time.
- codedokode 2y ago
  Maybe the solution is to have LLM read all that and summarize? And of course LLM can detect duplicated content - even if it is rewritten, it should generate a similar meaning vector.
- taneq 2y ago
  > seems like almost every article on the internet repeats itself in some form at least three times Spaced repetition, apparently it 'aids retention'. It's the same reason TV ads now run with a second shorter ad for the same thing later in the ad break.
- watwut 2y ago
  I used to write blog and stopped as google algoritms changed. Basically, if you do not do SEO with artificial constructions, you was as good as not existing. I could not find my own post in search even when googling the exact topic. I suspect that not coincidentally, blogosphere as used to be got much much smaller. Writing is not rewarding when no one finds it, so people just dont do it. I think that AI will make it even worst. When everyone fires ChatGPT to get compilation of stuff, there is even less reason to write. So you dont.
dgb23 2y ago
I always recommend SMEs to not use _any_ form of tracking/analytics by default. Just make the site useful, informative and well presented. To find out how users interact with it: Ask them, and/or do tests. Use the actual relationship with users to improve the site. Even ignoring privacy concerns, this mostly avoids bad UX patterns like "cookie banners", makes the sites _significantly_ faster and lighter. You end up with something that is higher quality. Exceptions to the rule exist, but that is always my baseline recommendation.
- chuckadams 2y ago
  Sounds needlessly extreme. Sure, don’t warp your site to please the SEO algorithms, but don’t plug your ears and cover your eyes to the basic info you’re already getting in your access logs.

itronitron 2y ago

I'm not sure whether it's SEO or AI that is to blame but in the past few years it seems like almost every article on the internet repeats itself in some form at least three times. Most blog posts about a topic feel like they were written as a High School English project. It's actually quite astonishing how bad the internet has become at conveying information compared to what it could be doing. So I'm tossing around an idea in my head, of implementing a search engine that computes a 'redundancy penalty' on every web page it indexed. The measure of redundancy would need to be multi-scale and consider repeated phrases, points made, and paragraphs as well, so not necessarily a simple problem. If you think this is something worth pursuing just let me know by commenting below.

elevation 2y ago
Three thoughts: First, effective writing will almost always restate the main idea somehow; you'll want your algorithm to be more sophisticated than "demote content that seems to repeat itself." Second, your novel algorithm will be expensive to compute, so you'll need some other signals to filter out very low value pages before you pay to analyze others. Finally, your technique will be valuable as a ranking signal until you (or someone else) begins to use your algorithm to help steer non trivial purchasing decisions. At that point, someone will dress up their low-value (or negative value) content to appear "high value" to your algorithm and your output will be filled with spam. You can delay this effect by keeping your user-base small, so consider selling private access to institutions who want a unique view of the internet for research or intelligence purposes. Good luck!
- Nextgrid 2y ago
  > someone will dress up their low-value content to appear "high value" to your algorithm You are falling into the trap of thinking that Google and its competitors are victims of this. They are not. They are knowingly letting themselves get "exploited" because it turns out that those spam sites contain ads (that may be Google's) or analytics (that may be Google's), or make the search result page ads (from Google) look good in comparison and more likely to be clicked. The heuristic to detect and block spam in general is very simple. Spam is there to get you to either buy something (the signal is the presence of a credit card form, "buy" call to action, or links that ultimately lead to the former) or view ads (signal there is presence of ads). This would be very hard to game because those signals only work when they are visible (and obnoxious), which also makes them trivially detectable purely at the visual level by a crude classifier (trained on website screenshots with ads vs the same page with an ad blocker). The resulting penalty score can be used as a ranking signal, so that all other signals being equal, a result without ads would rank higher than the one with ads for a given query. This is not rocket science. The problem is that the mainstream search engines have the same business model as the spammers (and profit off each other), so there is no reason to suddenly slaughter the cash cow. The myth that search engines are "victims" to spammers/SEO provides very convenient plausible deniability so they have no reason to disprove that either.
  - Mindwipe 2y ago
    This scoring is awful. Literally what you have done here is incentivise sites from well funded bad actors, or sites that push more elaborate scams on elderly people. A plausible and sustainable funding model is a good sign for sites.
    - Nextgrid 2y ago
      It's about incentivizing sites to do what the user wants them to do. As a user, 99% of the time I am not out to buy something, so all sites that do try to sell me something are just wasting both my time and their server resources. This isn't a complete blanket ban on ads or the commercial web. If there isn't a profitable way to run a website that provides the content matching the search query, the ad-infested website will still come up first. But now, a hobbyist, non-profit or even commercial enterprise that doesn't directly sell to you has a chance to outrank the ad-infested garbage.
thwarted 2y ago
> almost every article on the internet repeats itself in some form at least three times Even videos like YouTube shorts, which are meant to be short, have this problem. They are seriously padded with repetition. Here's thing, have you seen this before? Few people have seen this. Do you know how it works? I'm going to show you. People are surprised when they find out how it works. But I explain it. There must not be stats on choosing "never suggest this channel again", I select that pretty consistently when I'm looking for short form content and these videos waste all that time.
codedokode 2y ago
Maybe the solution is to have LLM read all that and summarize? And of course LLM can detect duplicated content - even if it is rewritten, it should generate a similar meaning vector.
taneq 2y ago
> seems like almost every article on the internet repeats itself in some form at least three times Spaced repetition, apparently it 'aids retention'. It's the same reason TV ads now run with a second shorter ad for the same thing later in the ad break.
watwut 2y ago
I used to write blog and stopped as google algoritms changed. Basically, if you do not do SEO with artificial constructions, you was as good as not existing. I could not find my own post in search even when googling the exact topic. I suspect that not coincidentally, blogosphere as used to be got much much smaller. Writing is not rewarding when no one finds it, so people just dont do it. I think that AI will make it even worst. When everyone fires ChatGPT to get compilation of stuff, there is even less reason to write. So you dont.

dgb23 2y ago

I always recommend SMEs to not use _any_ form of tracking/analytics by default. Just make the site useful, informative and well presented. To find out how users interact with it: Ask them, and/or do tests. Use the actual relationship with users to improve the site. Even ignoring privacy concerns, this mostly avoids bad UX patterns like "cookie banners", makes the sites _significantly_ faster and lighter. You end up with something that is higher quality. Exceptions to the rule exist, but that is always my baseline recommendation.

chuckadams 2y ago
Sounds needlessly extreme. Sure, don’t warp your site to please the SEO algorithms, but don’t plug your ears and cover your eyes to the basic info you’re already getting in your access logs.