7 ms·

Discussing CNET’s content-pruning to improve SEO ranking

141 points by marban 3y ago

jwilk 3y ago
3 days ago: https://news.ycombinator.com/item?id=37068464 https://news.ycombinator.com/item?id=37068464 ("CNET is deleting old articles to try to improve its Google Search ranking", >500 comments)
- hitekker 3y ago
  Ouch. Google's SEO PR rep really messed up the damage control. So many non-answers and links that essentially say "I'm not an engineer but trust me, the engineers I talk with say you're all wrong."
  - jasongill 3y ago
    Welcome to Google's SEO PR strategy since their inception. If Google says to do something, you should try to do it. If they say NOT to do something, you should definitely do it.
lern_too_spel 3y ago
I'd be surprised if search engines don't penalize websites that 404 previously crawled pages. They'd want to avoid the risk of a page disappearing and causing a poor user experience.
- rpgbr 3y ago
  Not necessarily 404. They could redirect (301) to another, better ranked similar content.
  - lolinder 3y ago
    That doesn't work for a news site—unless they're doing a ton of duplicate stories for some reason, "similar" isn't good enough if someone is looking for an article to go with a specific headline they found through search.
- causality0 3y ago
  I don't know why you're being downvoted. Link rot is a cancer we should all work against.
  - ghaff 3y ago
    Websites get redesigned, companies go out of business, etc. It's a nice principle that links should live on forever, but it's not very practical for a decentralized web (which many profess to be in favor of).
    - _dain_ 3y ago
      >Websites get redesigned, companies go out of business these two things are not remotely the same. if you are redesigning your website then it's your responsibility to make the old links redirect to the new pages. if you don't do that, it's a broken redesign.
  - TZubiri 3y ago
    Yes. Things should not die, we should all live forever, as bits and bytes
- telmo 3y ago
  They clearly couldn't care less about user experience. This is all about selling ads. The web was a beautiful thing for a moment in time, but the rotting corpse that is left after "adtech" did its thing is an anti-user monstrosity, to a degree that is almost comical.
  - justrealist 3y ago
    > an anti-user monstrosity The web today has 6 billion users instead of maybe 50mm in the golden age you're reminiscing about.
    - callalex 3y ago
      Can that be attributed to advancements of the web, or just the incredible advancements and price drops in the devices used to access it?
- snowwrestler 3y ago
  It depends on the pages. Search engines actually reward sites for 404ing previously crawled pages, if the pages had little value in the search index. Whether that is done on purpose, or is an unintended side effect of other ranking decisions, is up for debate.
visarga 3y ago
> A news website is not a bonsai tree well said
- ethbr1 3y ago
  But if traffic originators (e.g. Google) have a bonsai section that comprises the front half of their results... you can't fault a lot of non-bonsai trees for considering how they might qualify. At the end of the day, this is Google's mess. The article summarizes the situation well (clarification added by me): >> I think CNET is making a mistake, but they’ve also identified a serious problem for the digital news ecosystem: Old content is a burden to carry [because it lacks a viable economic model], and intentionally or not, Google isn’t making that lift any easier.
  - kedean 3y ago
    > you can't fault a lot of non-bonsai trees for considering how they might qualify. I will 100% fault them for acting on it. Just like I'll fault a company for choosing terrible single use plastics that increase their bottom line a bit, and companies that use AI to reduce head count. You can disagree with a decision even if you understand why it was made.
    - pessimizer 3y ago
      Companies should be using AI to reduce head count whenever possible. Welfare should be provided by the state, not privatized. Productivity gains are good. What if people are put out of work by dropping single-use plastics?
- ghaff 3y ago
  The main argument here is probably that news sites should have some obligation to the historical record. Normal product companies should arguably archive old product documentation in some form but relatively few would argue that they shouldn't regularly prune marketing literature, whitepapers, and other information that gets stale and is in the way of finding current stuff.
- 3y ago
cfiggers 3y ago
Sure, and I agree. But also, CNET has gotten more attention because of this stunt than anything else in the last decade because people WON'T STOP TALKING ABOUT IT. So while this technique probably won't help anyone else the same way, for CNET specifically I bet this has absolutely increased their Google traffic.
- worrycue 3y ago
  Will it though? I don’t visit CNET and I still won’t. Google isn’t going to rank any of their articles higher.
  - janesvilleseo 3y ago
    It helps them by generating a ton of links and mentions which Google still uses as a signal for authority. Getting high quality links from all sorts of places around the web is worth its weight in gold. So with these new links it will help to improve their rankings for a wide variety of keywords.
- rpgbr 3y ago
  This isn't the kind of attention a (allegedly) journalism outlet benefits from.
- jader201 3y ago
  > people WON'T STOP TALKING ABOUT IT Maybe this is anecdotal on my part, but I’ve heard nothing else about this since the initial article [1], then this — by the same author. On the other hand, maybe it’s anecdotal on your part, and not really that many people care, besides maybe this one author. > has absolutely increased their Google traffic I really doubt it, unless you have something to back the claim up. But even if so, this type of attention won’t sustain, and long term, it will likely only hurt them. [1] https://news.ycombinator.com/item?id=37107149 https://news.ycombinator.com/item?id=37107149
loonginthetooth 3y ago
I don't know much about SEO or content pruning, so I make no judgment on the conclusion of this article. What I can say is that this is genuinely one of the worst-written pieces I've ever read. Nowhere does the author provide any sort of argument for their conclusion. At no point during that long, rambling rant did the author give a single reason why we should agree with them, nor did they propose alternative solutions to the problem that CNET is facing. If this were written for a high-school english class it would receive a failing grade.
- TZubiri 3y ago
  It also displays a hilarious real live instance of the metaforgotten trope. "A good marriage is like an orange"
- nocoiner 3y ago
  By any chance, did you stop reading at the transition that looks awfully like the beginning of an ad box or some sort of call to action for whatever they’re presumably selling? The article actually continues and somewhat substantiates it’s argument, but I agree if you read only what appears to be the main body of the article, there’s very little there. The whole thing isn’t terrific either, but the page layout further detracts.
phowat 3y ago
Even if this strategy is good in terms of seo, they could have just archived those articles on a different domain.
- layer8 3y ago
  I wonder if also adding a redirect would still incur an SEO penalty.
  - pineconewarrior 3y ago
    Yes, sortof. 'Crawl budget' issues.
- mminer237 3y ago
  Can they not just noindex them?
adzicg 3y ago
one aspect of this that's overlooked in the discussion is how google penalises pages for bad core web vitals. instead of marking a single page bad and reducing its ranking, google decides using some weird algorithm to bundle a bunch of pages together into a group and penalise them based on the average core vitals metrics in the group. this would explain easily why old content is not worth keeping around. older pages might not be optimised for current web vitals metrics, but they can bring down stuff you care about a lot. SEO is black magic and astrology, so there's a lot of guessing going around, but I think I can vouch for this as this particular issue happened to one of the sites I maintain, and it caused the homepage to lose lots of traffic because it was bundled as related with some ancient blog posts. the homepage performed quite well according to all the metrics, but old pages did not, and until Google started penalising groups last year it didn't make a difference to keep the old pages around. Since they started penalising random groups, something that's important can be affected by something old and irrelevant, and the feedback loop is very slow. It takes 28 days to get feedback from the web console about any potential changes to page performance, so I tried fixing the old pages in a few ways, then just gave up and deleted them. the average rose up, the penalty to the whole group was gone, and the homepage again started getting a decent amount of traffic.
- Zambyte 3y ago
  We should not be optimizing for search engines. We should be optimizing for humans. If Google wants to shoot themselves in the foot by refusing to show their users the content they're looking for, let them. Don't let Google drag you down with them by encouraging you to make your site worse.
  - thfuran 3y ago
    And how do you propose making that happen?
    - Zambyte 3y ago
      1. Identify a target audience 2. Identify what that audience wants (by asking them) 3. Build what the target audience asked for if you agree that it is a sensible thing to create. 4. Tell the target audience that you have built your thing. Note that "target audience" here is probably not Google, but on all 4 points, SEO replaces "target audience" with "Google". Pruning old information is not something that any group of humans have requested besides Google.
travisporter 3y ago
How the hell did we get here? Were Microsoft and oracle and all these super rich tech giants asleep when Google was eating their lunch? Facebook too. It’s nuts
- adventured 3y ago
  No, almost everybody saw Google rising. Their capture of the search market and the relatively quick translation of sales from that was a very prominent event in tech circles. And Facebook wasn't relevant during the initial establishment and rise of Google (Facebook's first billion dollar sales year was 2010, 12 years after the founding of Google). Steve Ballmer (then CEO of Microsoft) swore he'd kill Google and bury Eric Schmidt - this is 2005 or earlier. Google's sales for 2004 for reference were $3.18 billion (for Alphabet now it's $282 billion). https://arstechnica.com/information-technology/2005/09/1106/ https://arstechnica.com/information-technology/2005/09/1106/ https://www.cnet.com/tech/tech-industry/court-docs-ballmer-vowed-to-kill-google/ https://www.cnet.com/tech/tech-industry/court-docs-ballmer-v...
- ranting-moth 3y ago
  I think Oracle is/was run by salespeople. They wanted you to pay a fortune for a database. Even if you could use a free database, they wanted you to pay a fortune for theirs. Because theirs could do a barrel roll and a hand-stand at the same time. Then of course when sales went down, you can always threat and extort your current customers: https://archive.is/Tbvw0 https://archive.is/Tbvw0 Microsoft just wanted to protect their windows thingy and pretend the internet didn't exists. They couldn't so they tried to embugger it. Once they had dominating market share with IE6, they simply said they weren't making money on IE and weren't going to update it any more. So they left it bug ridden for 5 years until the next release. Google shows up, with all their bondage and evil intentions carefully wrapped in satin and free beer on tap. But now the beer is finished and the satin is worn out. Everybody is hungover and tied up. The sounds coming from the next room is like something is being strapped on.
- Karellen 3y ago
  The thing is, even though I distrust Google a lot, I distrust Microsoft, Oracle and Facebook even less. Even if Bing got 2x better than Google at being able to show me what I actually wanted to find, I'd probably prefer Google for quite some time before even thinking of giving Bing a shot. For all I know, they do have better search engines than Google right now. But also, fuck those guys.

141 points by marban 3y ago

3 days ago: https://news.ycombinator.com/item?id=37068464 https://news.ycombinator.com/item?id=37068464 ("CNET is deleting old articles to try to improve its Google Search ranking", >500 comments)

hitekker 3y ago
Ouch. Google's SEO PR rep really messed up the damage control. So many non-answers and links that essentially say "I'm not an engineer but trust me, the engineers I talk with say you're all wrong."
- jasongill 3y ago
  Welcome to Google's SEO PR strategy since their inception. If Google says to do something, you should try to do it. If they say NOT to do something, you should definitely do it.

lern_too_spel 3y ago

I'd be surprised if search engines don't penalize websites that 404 previously crawled pages. They'd want to avoid the risk of a page disappearing and causing a poor user experience.

rpgbr 3y ago
Not necessarily 404. They could redirect (301) to another, better ranked similar content.
- lolinder 3y ago
  That doesn't work for a news site—unless they're doing a ton of duplicate stories for some reason, "similar" isn't good enough if someone is looking for an article to go with a specific headline they found through search.
causality0 3y ago
I don't know why you're being downvoted. Link rot is a cancer we should all work against.
- ghaff 3y ago
  Websites get redesigned, companies go out of business, etc. It's a nice principle that links should live on forever, but it's not very practical for a decentralized web (which many profess to be in favor of).
  - _dain_ 3y ago
    >Websites get redesigned, companies go out of business these two things are not remotely the same. if you are redesigning your website then it's your responsibility to make the old links redirect to the new pages. if you don't do that, it's a broken redesign.
- TZubiri 3y ago
  Yes. Things should not die, we should all live forever, as bits and bytes
telmo 3y ago
They clearly couldn't care less about user experience. This is all about selling ads. The web was a beautiful thing for a moment in time, but the rotting corpse that is left after "adtech" did its thing is an anti-user monstrosity, to a degree that is almost comical.
- justrealist 3y ago
  > an anti-user monstrosity The web today has 6 billion users instead of maybe 50mm in the golden age you're reminiscing about.
  - callalex 3y ago
    Can that be attributed to advancements of the web, or just the incredible advancements and price drops in the devices used to access it?
snowwrestler 3y ago
It depends on the pages. Search engines actually reward sites for 404ing previously crawled pages, if the pages had little value in the search index. Whether that is done on purpose, or is an unintended side effect of other ranking decisions, is up for debate.

visarga 3y ago

> A news website is not a bonsai tree well said

ethbr1 3y ago
But if traffic originators (e.g. Google) have a bonsai section that comprises the front half of their results... you can't fault a lot of non-bonsai trees for considering how they might qualify. At the end of the day, this is Google's mess. The article summarizes the situation well (clarification added by me): >> I think CNET is making a mistake, but they’ve also identified a serious problem for the digital news ecosystem: Old content is a burden to carry [because it lacks a viable economic model], and intentionally or not, Google isn’t making that lift any easier.
- kedean 3y ago
  > you can't fault a lot of non-bonsai trees for considering how they might qualify. I will 100% fault them for acting on it. Just like I'll fault a company for choosing terrible single use plastics that increase their bottom line a bit, and companies that use AI to reduce head count. You can disagree with a decision even if you understand why it was made.
  - pessimizer 3y ago
    Companies should be using AI to reduce head count whenever possible. Welfare should be provided by the state, not privatized. Productivity gains are good. What if people are put out of work by dropping single-use plastics?
ghaff 3y ago
The main argument here is probably that news sites should have some obligation to the historical record. Normal product companies should arguably archive old product documentation in some form but relatively few would argue that they shouldn't regularly prune marketing literature, whitepapers, and other information that gets stale and is in the way of finding current stuff.
3y ago

cfiggers 3y ago

Sure, and I agree. But also, CNET has gotten more attention because of this stunt than anything else in the last decade because people WON'T STOP TALKING ABOUT IT. So while this technique probably won't help anyone else the same way, for CNET specifically I bet this has absolutely increased their Google traffic.

worrycue 3y ago
Will it though? I don’t visit CNET and I still won’t. Google isn’t going to rank any of their articles higher.
- janesvilleseo 3y ago
  It helps them by generating a ton of links and mentions which Google still uses as a signal for authority. Getting high quality links from all sorts of places around the web is worth its weight in gold. So with these new links it will help to improve their rankings for a wide variety of keywords.
rpgbr 3y ago
This isn't the kind of attention a (allegedly) journalism outlet benefits from.
jader201 3y ago
> people WON'T STOP TALKING ABOUT IT Maybe this is anecdotal on my part, but I’ve heard nothing else about this since the initial article [1], then this — by the same author. On the other hand, maybe it’s anecdotal on your part, and not really that many people care, besides maybe this one author. > has absolutely increased their Google traffic I really doubt it, unless you have something to back the claim up. But even if so, this type of attention won’t sustain, and long term, it will likely only hurt them. [1] https://news.ycombinator.com/item?id=37107149 https://news.ycombinator.com/item?id=37107149

loonginthetooth 3y ago

I don't know much about SEO or content pruning, so I make no judgment on the conclusion of this article. What I can say is that this is genuinely one of the worst-written pieces I've ever read. Nowhere does the author provide any sort of argument for their conclusion. At no point during that long, rambling rant did the author give a single reason why we should agree with them, nor did they propose alternative solutions to the problem that CNET is facing. If this were written for a high-school english class it would receive a failing grade.

TZubiri 3y ago
It also displays a hilarious real live instance of the metaforgotten trope. "A good marriage is like an orange"
nocoiner 3y ago
By any chance, did you stop reading at the transition that looks awfully like the beginning of an ad box or some sort of call to action for whatever they’re presumably selling? The article actually continues and somewhat substantiates it’s argument, but I agree if you read only what appears to be the main body of the article, there’s very little there. The whole thing isn’t terrific either, but the page layout further detracts.

phowat 3y ago

Even if this strategy is good in terms of seo, they could have just archived those articles on a different domain.

layer8 3y ago
I wonder if also adding a redirect would still incur an SEO penalty.
- pineconewarrior 3y ago
  Yes, sortof. 'Crawl budget' issues.
mminer237 3y ago
Can they not just noindex them?

adzicg 3y ago

one aspect of this that's overlooked in the discussion is how google penalises pages for bad core web vitals. instead of marking a single page bad and reducing its ranking, google decides using some weird algorithm to bundle a bunch of pages together into a group and penalise them based on the average core vitals metrics in the group. this would explain easily why old content is not worth keeping around. older pages might not be optimised for current web vitals metrics, but they can bring down stuff you care about a lot. SEO is black magic and astrology, so there's a lot of guessing going around, but I think I can vouch for this as this particular issue happened to one of the sites I maintain, and it caused the homepage to lose lots of traffic because it was bundled as related with some ancient blog posts. the homepage performed quite well according to all the metrics, but old pages did not, and until Google started penalising groups last year it didn't make a difference to keep the old pages around. Since they started penalising random groups, something that's important can be affected by something old and irrelevant, and the feedback loop is very slow. It takes 28 days to get feedback from the web console about any potential changes to page performance, so I tried fixing the old pages in a few ways, then just gave up and deleted them. the average rose up, the penalty to the whole group was gone, and the homepage again started getting a decent amount of traffic.

Zambyte 3y ago
We should not be optimizing for search engines. We should be optimizing for humans. If Google wants to shoot themselves in the foot by refusing to show their users the content they're looking for, let them. Don't let Google drag you down with them by encouraging you to make your site worse.
- thfuran 3y ago
  And how do you propose making that happen?
  - Zambyte 3y ago
    1. Identify a target audience 2. Identify what that audience wants (by asking them) 3. Build what the target audience asked for if you agree that it is a sensible thing to create. 4. Tell the target audience that you have built your thing. Note that "target audience" here is probably not Google, but on all 4 points, SEO replaces "target audience" with "Google". Pruning old information is not something that any group of humans have requested besides Google.

travisporter 3y ago

How the hell did we get here? Were Microsoft and oracle and all these super rich tech giants asleep when Google was eating their lunch? Facebook too. It’s nuts

adventured 3y ago
No, almost everybody saw Google rising. Their capture of the search market and the relatively quick translation of sales from that was a very prominent event in tech circles. And Facebook wasn't relevant during the initial establishment and rise of Google (Facebook's first billion dollar sales year was 2010, 12 years after the founding of Google). Steve Ballmer (then CEO of Microsoft) swore he'd kill Google and bury Eric Schmidt - this is 2005 or earlier. Google's sales for 2004 for reference were $3.18 billion (for Alphabet now it's $282 billion). https://arstechnica.com/information-technology/2005/09/1106/ https://arstechnica.com/information-technology/2005/09/1106/ https://www.cnet.com/tech/tech-industry/court-docs-ballmer-vowed-to-kill-google/ https://www.cnet.com/tech/tech-industry/court-docs-ballmer-v...
ranting-moth 3y ago
I think Oracle is/was run by salespeople. They wanted you to pay a fortune for a database. Even if you could use a free database, they wanted you to pay a fortune for theirs. Because theirs could do a barrel roll and a hand-stand at the same time. Then of course when sales went down, you can always threat and extort your current customers: https://archive.is/Tbvw0 https://archive.is/Tbvw0 Microsoft just wanted to protect their windows thingy and pretend the internet didn't exists. They couldn't so they tried to embugger it. Once they had dominating market share with IE6, they simply said they weren't making money on IE and weren't going to update it any more. So they left it bug ridden for 5 years until the next release. Google shows up, with all their bondage and evil intentions carefully wrapped in satin and free beer on tap. But now the beer is finished and the satin is worn out. Everybody is hungover and tied up. The sounds coming from the next room is like something is being strapped on.
Karellen 3y ago
The thing is, even though I distrust Google a lot, I distrust Microsoft, Oracle and Facebook even less. Even if Bing got 2x better than Google at being able to show me what I actually wanted to find, I'd probably prefer Google for quite some time before even thinking of giving Bing a shot. For all I know, they do have better search engines than Google right now. But also, fuck those guys.