9 ms·

Some thoughts about The Verge article on SEO

72 points by bhartzer 3y ago

nickpeterson 3y ago
Yikes, this comes across as extremely defensive.
- ffhhhgvvbb 3y ago
  [dead]
- JohnFen 3y ago
  Well, it is defensive because he's literally defending himself. I don't think it deserves a "yikes", though. His tone is calm, his arguments cogent, etc. I think this was actually a very good response.
baxtr 3y ago
For those wondering, here’s The Verge article: Title: “The people who ruined the internet” https://www.theverge.com/features/23931789/seo-search-engine-optimization-experts-google-results https://www.theverge.com/features/23931789/seo-search-engine...
- tqi 3y ago
  Of course an article titled pointing fingers at others for ruining the internet would have an outbrain chumbox at the bottom...
- dang 3y ago
  It's here, with a debaited title: As the public begins to believe Google isn’t as useful, what happens to SEO? - https://news.ycombinator.com/item?id=38097938 https://news.ycombinator.com/item?id=38097938 - Nov 2023 (12 comments)
  - baxtr 3y ago
    In this case, I think it’s valuable to know what the original title is since it conveys the spirit of the article.
DwnVoteHoneyPot 3y ago
I tried reading both the Verge and this article and 10 paragraphs into each, still in the preamble.
- ffhhhgvvbb 3y ago
  [dead]
parraccourci 3y ago
Is there a tldr of the original article and the reaction(s)?
- JohnFen 3y ago
  Extreme tl;dr incoming... Original article: Google search sucks now Response article: No, it doesn't
  - parraccourci 3y ago
    Thanks mate
xpose2000 3y ago
I think Danny should avoid future interviews with the Verge. heh. But seriously, I just don't like how the tone of the interview came across. Does anyone see Danny yelling in these conversations to warrant the use of the exclamation mark? If anything this makes the Verge look bad. As for addressing SEO concerns. There is a lot of frustration out there these days by small sites and companies trying to make their way and Google results can be hit or miss. Major publications like CNET, Forbes, CNN etc are purposefully creating content and cramming it with affiliate links to sell crap to the masses. When a major publication writes about something its not an expert in, one has to start to raise eyebrows and wonder... Its painfully obvious. They get away with it because they are huge brands and can rank for anything, so they are abusing their power. Sean Kaye says it best here: https://twitter.com/SeanDoesLife/status/1716935563075559630 https://twitter.com/SeanDoesLife/status/1716935563075559630 Additionally, I want to mention an obvious manipulative practice that companies seem to be rewarded for, when if anything, should be penalized for. And that is avoiding the "standard news" syntax of published content via manipulative URLS. Namely, avoiding using /id/date/title-of-post (or something similar) and just using the rootdomain/title-of-post to make it rank higher and seem more important than it is. These pages are not an About page, or Privacy Page, or Terms of Service page. Its manipulation and a shady practice and companies should be penalized for it.
- Terr_ 3y ago
  > Namely, avoiding using /id/date/title-of-post (or something similar) and just using the rootdomain/title-of-post to make it rank higher and seem more important than it is. This causes the small wayward fragments of Library Science curriculum embedded in my brain to quiver with rage. Bonus points if the tail end of the URL contains what may-or-may-not be a bunch of tracking shit and it's not obvious how much it can be shortened without breaking the link.
- nickpeterson 3y ago
  I read the verge and listen to their podcast. They don’t come across as overly sensationalist and seem pretty fair in other interviews. I think Google knows their search product isn’t as useful as it once was. Not sure about the root causes, but that’s just my impression from using it for 15 years.
Sloppy 3y ago
[flagged]
goodroot 3y ago
Danny is clearly upset, and I would be too. We all love the Internet. Imagine being fingered in an article as some mad SEO guy who is among "the people who ruined the Internet". The Verge is a big platform... Also, SEO is of the more voodoo & charlatan filled branches of technical esoterica. Very little of it is falsifiable or clear. And there is quackery as far as the eye can see. Transparency into search algos would be better for us all. But the cost of algo transparency is transparency into adtech. And that's a hill Google will die on, and why we need alternatives.
- shadowgovt 3y ago
  > Transparency into search algos would be better for us all I think that the jury's out on that topic. Danny's absolutely right in asserting that if the full algo were known, people would write to optimize against the algo, which would defeat the purpose of the algo. Goodhart's Law is in full effect: “When a measure becomes a target, it ceases to be a good measure."
  - pseg134 3y ago
    So then the algorithm is crap and that is why they don’t want to show it. If the “full algo” actually prioritizes high quality content we would certainly want everyone to optimize against it!
    - shadowgovt 3y ago
      In general, these models are approximations of an ideal, or some kind of statistical summary across systems that are too complex to completely model. There's a wide gap between "this algorithm is crap" and "this algorithm stops working if we publish the whole thing publicly and people can explicitly tune data to make number-go-up." That's like claiming a machine learning algorithm is crap because it's possible to build bespoke counter-inputs that maximize badness in the output; that's possible with most ML algorithms, but when someone's not trying to break the machine on purpose, those algorithms often work great.
      - 3y ago
avarun 3y ago
The original Verge article is hilariously bad. It somehow manages to provoke false rage at Google while attempting to sympathize with every actual charlatan that's the reason for SEO spam on the internet. The Verge needs to look at problems with better lenses than "big company bad, individual people good". But then again, when all you have is a hammer I guess everything looks like a nail.
rtsil 3y ago
> But the point – and my frustration with reporters who sometimes ask me this type of question – is that it’s clearly absurd. They themselves use Google all the time to find things. Millions of people each day clearly find information that’s useful. And yet, I’ve repeatedly seen notable tech journalists with a straight-face share how you can’t find anything on Google, that it is useless and so on. Google isn't completely useless, but it has became extremely painful to use. There was a time when Google returned exactly what I needed, or said that there's no result. For the last 5 years or so, I have to manually filter the results and, back when pagination still existed, I went directly to page 2 on some searches to skip the first dozen SEO-filled results. So, sure, I keep finding useful results on Google, but only after having to navigate through a sea of SEO-optimized links. And yes, I still keep using Google for a third of my search because I have no better choice on these topics.
- drcongo 3y ago
  Kagi. Better results than Google ever returned, and infinitely better than the GPT generated spam you get on the first several pages of Google these days.
  - ipaddr 3y ago
    How does it rank nsfw content?
    - JohnFen 3y ago
      Give it a try. They have a free tier that is generous enough to find out if the service works well for you.
- visarga 3y ago
  > There was a time when Google returned exactly what I needed, or said that there's no result. For the last 5 years or so, I have to manually filter the results Sounds to me like Google search is also suffering from a lot of hallucinations, as if progress went backwards.
stillwithit 3y ago
The Verge is rent seeking; they need to be discoverable to exist. The Verge is like many other web assets these days; facing extinction and mad at everyone else over it. What a shock, past human social pageantry is being obsoleted. Thats never happened before; we all speak Latin still, and serve the British monarchy, don’t you know. Horse and buggy, rotary phone makers started a support group all the wannabe Lois Lanes and Clark Kent’s can join up with.
zenincognito 3y ago
I have been in this industry just 5 years shy of Danny. Danny was a big critic of Google back in the days. What he doesn't realise is that Google has deployed the oldest trick in the book. They took the most prominent critic in the industry and made him into their spokesman. Effectively, leveraging a lot of goodwill he holds in the industry into their own. I am not saying what Danny is saying is untrue or has malice. I am just saying that Google search has deteriorated immensely and is skewed to about 100 sites that dominate it competely. There is very little space for small business or genuine knowledgespeak unless you have a big budget to churn our content. Just look at Google's testimony in the recent antitrust case and you can see what Google says via the liasion team is opposite to how SEO actually works. Ex - Google using click data but constantly saying for over a decade they don't use the click data.
- dannysullivan 3y ago
  I get this view. But as my post explains, I'd quit writing about search. I was done. There wasn't going to be more criticism (or praise or whatever) from me because I'd retired from writing about search. I didn't have plans to go to Google when I retired. No one there even knew I was leaving. Which ... you or anyone can choose to believe or not, but that's how it is. I was far from the only critic (or advocate) for Google or other search engines. There are plenty of others. New people, and with good views, continue to come into the space. The idea of "Google hired me to quiet me," again, while I get it, just wouldn't resolve that. By the way, nor would some alternative idea that I somehow had secret details of spamming techniques make sense, either. Google had and still has an excellent spam team. They didn't need me to come in and somehow fill gaps. What Google gained by me coming in, I hope, is someone that both tries to help people better understand how the search engine works from within the search quality team (that's where I work, in that team) and also bring back into that team advocacy and feedback from the outside world (which typically, I realize, isn't that clear to those people outside Google -- here's an example I shared of this last week when asked: https://twitter.com/searchliaison/status/1720491595420856329 https://twitter.com/searchliaison/status/1720491595420856329 )
ChrisArchitect 3y ago
Related thread on the Verge article from earlier: https://news.ycombinator.com/item?id=38097938 https://news.ycombinator.com/item?id=38097938
micromacrofoot 3y ago
This is a case where I think both The Verge and Danny get things wrong, they're both far too biased. The Verge has come to some forgone conclusion about the decline of Google, and Danny works for Google.
Apreche 3y ago
> I almost certainly would not have said that everyone “who tried to rank higher by messing with the algorithm would be blocked.” That’s not a direct quote, but it also doesn’t make sense as a paraphrase. > We have ranking systems that largely do not “block” content. They look to reward good content and show it highly in results; lower-quality content drops. Even with actual spam, it’s unusual that we’d literally “block” an entire site. It just isn’t likely to rank well for general queries that aren’t somehow very specific to it. This, to me, is the fundamental problem that is ruining Google search results. There are many many sites that absolutely should be completely blocked. But Google will never block or downrank those sites because those sites are their golden goose. We've all done a search with a programming question, expecting the top result to either be the official documentation or Stack Overflow. Instead, we get a content farm site that has either stolen, or ML-generated, the answers from other sites that were actually responsible for crafting the content in the first place. You know the sites, I don't have to name them here. These content farm sites not only stole their content, but they are very unpleasant to visit. Sure, they may load fast. They may have the content you searched for. But they also have ads. Lots of ads. They also have ad blocker blockers. And they have little pop-ups that ask you to subscribe to the newsletter. Those sorts of things make a site unpleasant to visit. Those are the sorts of things that should make a site get heavily downranked, or completely blocked. No matter what content such a site has, a human doesn't want to suffer through the experience of going to that site. But Google can't do that. Those sites are their golden goose. Those sites are serving up ads powered by Google. If Google downranks sites that are a bad unpleasant ad-filled user experience, and upranks ad-free wonderful web sites, they're hosed. All those ad impressions their customers paid for will never happen if traffic to ad-laden web sites plummets. TL;DR: The ideal search engine would only allow a site to show up in a search result if that site was completely free of all ads, trackers, paywalls, pop-ups, or any other unpleasant similar thing. The entire rest of the web would be de-indexed. A search engine that itself is paid for by ads can never do that.
- JohnFen 3y ago
  > This, to me, is the fundamental problem that is ruining Google search results. That's certainly a large factor, but I think a greater factor (at least for me) is Google's use of ML to try to interpret what I "really" want when I write a search query instead of just going by what I actually wrote. Google makes terrible guesses as to my intent.
JohnMakin 3y ago
You will never, ever, ever, ever convince me that search is "better" now than it ever has been, or even more than a few years ago. This seems completely delusional to me. I define good search as me being able to quickly find what I am looking for. Approximately 1-2 years ago, I forget when it started, I noticed weird instances where no matter what I put into the engine and how long I scrolled or clicked around I absolutely could not find what I was looking for and gave up. I remember vividly the first time it happened, because it startled me. Google had never been like that. At first it was rare, then more often, these days it became so unusable I had to put it down. I think my final breaking point was trying to find the score of a sports game that had just happened a few hours before and being completely unable to. That's unimportant info, but since I frequently used google often in my day-to-day job, the inability to find what I need is a real serious issue. Bing has been great so far.
- JohnMakin 3y ago
  Good example and fun exercise for anyone to try - input to google search "Migrating off of GCP." You will only find documentation and videos how to migrate TO GCP. Yea, I don't buy that google mysteriously forgot how the english language works, sorry.
  - cma 3y ago
    Google never worked well with that kind of language processing task in the heyday of its usefulness either. Searching for "dresses that aren't red" in image search returned almost 100% red dress images (maybe better now). Extremely common words like "off" and "of" were essentially ignored.
    - JohnMakin 3y ago
      You can try the same experiment with any combination of words, the results are basically the same.
    - johnnyanmac 3y ago
      I think the worse part is that there were tools around this and they also don't work anymore. Try to search "football -American" and you'll still end up with the search for American Football. search "football -nfl" and it will instead try to inject news of local college/high school teams instead of resorting to FIFA or other assossiations for the worldwide term. I never expected such boolean logic to be perfect, but it barely even seems to try anymore.
mediumsmart 3y ago
I don't know what the Verge is and I don't know much about SpamEngineOptimisation but I think that its not that complicated. You pay Google to have your rubbish rank higher and you pay Kagi to not have to look at that when searching for something useful. And there is still room for beautiful blogposts, interviews and discussions about the different angles and what the real issue is and how to look at this in a calm and objective way and what solutions might be on the horizon, hey, go for it. Choose a nice webfont and color palette and knock yourselves out. toodles.
nix0n 3y ago
> I do recall one question [...] along the lines of “So, what do you think about the idea that people can’t find things on Google Search,” to which I recall joking something like, “Yes, clearly no one finds things any more.” I really, actually, do not find things on Google anymore. The rate of success has gotten quite bad, under 50%, when it used to be closer to 95%. Sullivan says in his article that his role is "part of the search quality team", but he thinks this decline in quality is a laughing matter. It's not going to get better.
- whstl 3y ago
  I'm using ChatGPT for 90% of documentation search work, hallucinations and all, simply because Google has become THAT worse for development IMO. I can't find shit anymore. Sometimes when I have to use a specific library, I just download the documentation and search using VSCode or something.
  - pastamania 3y ago
    I'm largely mystified at a lot of the 'Google is rubbish now' takes, but dev documentation is one of the areas that I genuinely find Google was better at a few years ago. I actually do tend to get answers to my questions, but a lot of them are desperately outdated and no longer relevant. My take on what happened is that Google has a metric in their algorithms called 'Query deserves freshness' - basically, identifying a search where the user isn't going to want old information, and artifically promoting more recent pages over better linked to but older documents. If someone wants to see the weather, they probably aren't looking for the weather on an extremely weird day 4 years ago just because a lot of people linked to it back then. They seem to have reduced that down, I suspect to reduce search being flooded with misinformation during the pandemic when all information was recent and the more controversial takes sometimes got more coverage. As an unexpected result, if I'm searching a dev query, it ends up surfacing a ton of results from years ago. Yeah, those pages may have more links and history and probably seem to an algorithm to be more trustworthy, but if it's for a library 20 versions old that long since stopped working the way the stackoverflow answer from 2011 says it did, that's pretty useless. Still, as a trade off, probably better me having to use the time filter dropdown to limit results to recent one's than someone necking bleach and horse de-wormer when they get Covid. And while I'm not quite as onboard with GPT being the death of search as some, dev documentation is one of those areas where generative AI is genuinely a better tool than web search ever was.
doogerdog 3y ago
The verge produces some good articles. This is not an example. I tried to read it, but quickly got bogged down in irrelevant fluff. So I skimmed the rest. Then I searched the article for "Sullivan". Every other occurrence is followed by, "is mad" "is pissed" "impatient corporate stooge", and other improbable attacks. Wow! She wrote a metric ton of words just to say, "I hate Danny". It is true that Google is a swamp. Thank goodness for Kagi.
lucubratory 3y ago
This whole article feels like Danny Sullivan cooling off and patiently trying to explain that no, we're all wrong, the sky really is red. It just doesn't fit with my experience. 60% of the time, for actual searches (not just using Google as my lazy equivalent of a memorised bookmarks bar because bookmarks don't work well on mobile) don't work, and I have to wade through crap that is mentally tiring to deal with just to be given the privilege of discovering that it didn't work, the information I want is not being provided by this search, or the toll being exacted by SEO spam crap for the information is soul destroying enough that I need to go somewhere else. As just one example, my husband likes to play Minecraft. Sometimes he asks me to look up something while he's playing. A couple of weeks ago he asked me several questions about a creature called a "Vex". This is what happened when I tried to use Google to answer those questions. One of the questions was about its health on Hard difficulty, another was if there were any blocks it couldn't phase through, another was if it's going to attack the villagers he's trying to keep safe in this little area. I don't want to search all of these individually and I figure all of these answers probably exist on one wiki page, so I search "Minecraft Vex wiki". The first result is a Fandom link; click on that, immediate banner scam ad covering half the screen, pop-up scam ad on the bottom, a third scam ad doing some weird scrolling thing on the side of the screen, that turns into an auto-playing video. One of the ads features a group of sexualised women selling deodorant in a tropical environment. The auto-playing video appears to be trying to tell me how great it would be to have a beer (I am 7 years sober, and I've told Google not to give me ads about alcohol). It has been approximately five seconds since I hit the search button on "Minecraft Vex wiki". It is a failure on Google's part that the first result is a site that has this experience. The second result is one of those IGN "wikis" that are generally worse than any other option and also full of ads, so I'm not interested but click to confirm - okay, this looks like it was written by a bad teenager, someone from Fiverr, or both. And it's absolutely full of ads, time to swipe back. The next thing in the results is four variations on questions about Vex, none of them being the questions I had. The next thing in the results is eight little thumbnails directly in the search results, with a little icon in the corner of each image indicating the website it's coming from - why would these image thumbnails ever, ever pop up on what is clearly a search for detailed, technical information? I didn't run an image search, I know how to click the image search button if I want that, and what I was looking for wasn't even possible to solve with an image thumbnail. They don't even have any text to tell me whether they're worth clicking on, they are a completely useless result for my search. What's worse, between them and the useless questions that I didn't ask, this takes up maybe five results worth of space. By the time I get to the third result on the page, I am maybe a third of the way down the page and it's the official Minecraft announcement page for the Vex mob. This is basically a good result for the Minecraft and Vex portions of the term (third time's the charm), but it still doesn't have the information I'm looking for, it's a general overview that doesn't answer any of the specific questions I had. Every other result from here on out was worse than this, until the result that was second to last. That was a minecraft.wiki link which wasn't torturous to navigate, and after clicking on it it was able to answer all of my questions, after I clarified some things with my husband. Why wasn't this result first? "Just make a good website" does not, in any way, describe what Fandom is providing, and Google recommends it to me as the first option. I don't know if it's specific to me because of my search history now, but conducting the same search now has the minecraft.wiki link as the first result, so it's clear Google is trying something to improve. I have zero confidence that the next time I search for information whose location I don't already know, that it will be any different. This is how at least half of my attempts to use Google to actually search go, and this was an absolute best case scenario for search because it was about probably the world's most popular video game with extensive technical documentation in existence, and I knew exactly what I wanted (a good wiki page on the Vex from Minecraft) and put it in the search, and because Google actually gave me the answer eventually instead of just crapping the bed and having nothing, like it often does. The entire experience was onerous and visually/auditorily offensive drudgery, so that I could answer someone's questions about a video game while they were focusing on another resource they were using to build something in the game. Does Google mark this as a "success" because I got the information I wanted? Because it wasn't successful, it made me want to gouge my eyes out just like every other time I use Google, and what it made me want most of all was an LLM from five years in the future that can just field those questions and provide answers without all of this... "experience". This was not my experience Googling questions 10 years ago. I do not know or care if this is all Danny Sullivan's fault, I don't know him from a can of bread, but it's someone's fault and frankly I hope the company that does this crap can't adapt fast enough and gets destroyed by AI. And you know what's funny? Before my first time using an LLM, I had just accepted that every time I was going to search on the internet it would be a little more soul destroying than it was before, every day for the rest of my life, and there was nothing I could ever do about it because no one would ever be able to compete with Google's monopoly. I had no anger because I had no hope that a solution to this was even possible. After my first time using an LLM with internet access, there were a boatload of problems and potential for abuse but I could immediately see that this technology could insulate me from the demons on the internet that make it a soul-trampling experience, and at least if it got the answers wrong it could say sorry instead of exposing me to unwanted pictures of a woman in a bikini covered in beer foam. Google is a bad, bad, bad experience, and the fact the SEO spammers and this Google employee think it's very effective is both unsurprising and unconvincing. Don't even get me started on the actual Google ads and their determination to make me read the Epoch Times, a newspaper run by a cult that my friend barely escaped from and which there's apparently no way to inform Google I am never going to click on so please don't show it to me anymore. Okay. I got mad but with reason, Google is rage-inducing in the modern era. Now I can move on with my day.

72 points by bhartzer 3y ago

nickpeterson 3y ago

Yikes, this comes across as extremely defensive.

ffhhhgvvbb 3y ago
[dead]
JohnFen 3y ago
Well, it is defensive because he's literally defending himself. I don't think it deserves a "yikes", though. His tone is calm, his arguments cogent, etc. I think this was actually a very good response.

baxtr 3y ago

For those wondering, here’s The Verge article: Title: “The people who ruined the internet” https://www.theverge.com/features/23931789/seo-search-engine-optimization-experts-google-results https://www.theverge.com/features/23931789/seo-search-engine...

tqi 3y ago
Of course an article titled pointing fingers at others for ruining the internet would have an outbrain chumbox at the bottom...
dang 3y ago
It's here, with a debaited title: As the public begins to believe Google isn’t as useful, what happens to SEO? - https://news.ycombinator.com/item?id=38097938 https://news.ycombinator.com/item?id=38097938 - Nov 2023 (12 comments)
- baxtr 3y ago
  In this case, I think it’s valuable to know what the original title is since it conveys the spirit of the article.

DwnVoteHoneyPot 3y ago

I tried reading both the Verge and this article and 10 paragraphs into each, still in the preamble.

ffhhhgvvbb 3y ago
[dead]

parraccourci 3y ago

Is there a tldr of the original article and the reaction(s)?

JohnFen 3y ago
Extreme tl;dr incoming... Original article: Google search sucks now Response article: No, it doesn't
- parraccourci 3y ago
  Thanks mate

xpose2000 3y ago

I think Danny should avoid future interviews with the Verge. heh. But seriously, I just don't like how the tone of the interview came across. Does anyone see Danny yelling in these conversations to warrant the use of the exclamation mark? If anything this makes the Verge look bad. As for addressing SEO concerns. There is a lot of frustration out there these days by small sites and companies trying to make their way and Google results can be hit or miss. Major publications like CNET, Forbes, CNN etc are purposefully creating content and cramming it with affiliate links to sell crap to the masses. When a major publication writes about something its not an expert in, one has to start to raise eyebrows and wonder... Its painfully obvious. They get away with it because they are huge brands and can rank for anything, so they are abusing their power. Sean Kaye says it best here: https://twitter.com/SeanDoesLife/status/1716935563075559630 https://twitter.com/SeanDoesLife/status/1716935563075559630 Additionally, I want to mention an obvious manipulative practice that companies seem to be rewarded for, when if anything, should be penalized for. And that is avoiding the "standard news" syntax of published content via manipulative URLS. Namely, avoiding using /id/date/title-of-post (or something similar) and just using the rootdomain/title-of-post to make it rank higher and seem more important than it is. These pages are not an About page, or Privacy Page, or Terms of Service page. Its manipulation and a shady practice and companies should be penalized for it.

Terr_ 3y ago
> Namely, avoiding using /id/date/title-of-post (or something similar) and just using the rootdomain/title-of-post to make it rank higher and seem more important than it is. This causes the small wayward fragments of Library Science curriculum embedded in my brain to quiver with rage. Bonus points if the tail end of the URL contains what may-or-may-not be a bunch of tracking shit and it's not obvious how much it can be shortened without breaking the link.
nickpeterson 3y ago
I read the verge and listen to their podcast. They don’t come across as overly sensationalist and seem pretty fair in other interviews. I think Google knows their search product isn’t as useful as it once was. Not sure about the root causes, but that’s just my impression from using it for 15 years.

Sloppy 3y ago

[flagged]

goodroot 3y ago

Danny is clearly upset, and I would be too. We all love the Internet. Imagine being fingered in an article as some mad SEO guy who is among "the people who ruined the Internet". The Verge is a big platform... Also, SEO is of the more voodoo & charlatan filled branches of technical esoterica. Very little of it is falsifiable or clear. And there is quackery as far as the eye can see. Transparency into search algos would be better for us all. But the cost of algo transparency is transparency into adtech. And that's a hill Google will die on, and why we need alternatives.

shadowgovt 3y ago
> Transparency into search algos would be better for us all I think that the jury's out on that topic. Danny's absolutely right in asserting that if the full algo were known, people would write to optimize against the algo, which would defeat the purpose of the algo. Goodhart's Law is in full effect: “When a measure becomes a target, it ceases to be a good measure."
- pseg134 3y ago
  So then the algorithm is crap and that is why they don’t want to show it. If the “full algo” actually prioritizes high quality content we would certainly want everyone to optimize against it!
  - shadowgovt 3y ago
    In general, these models are approximations of an ideal, or some kind of statistical summary across systems that are too complex to completely model. There's a wide gap between "this algorithm is crap" and "this algorithm stops working if we publish the whole thing publicly and people can explicitly tune data to make number-go-up." That's like claiming a machine learning algorithm is crap because it's possible to build bespoke counter-inputs that maximize badness in the output; that's possible with most ML algorithms, but when someone's not trying to break the machine on purpose, those algorithms often work great.
    - 3y ago

avarun 3y ago

The original Verge article is hilariously bad. It somehow manages to provoke false rage at Google while attempting to sympathize with every actual charlatan that's the reason for SEO spam on the internet. The Verge needs to look at problems with better lenses than "big company bad, individual people good". But then again, when all you have is a hammer I guess everything looks like a nail.

rtsil 3y ago

> But the point – and my frustration with reporters who sometimes ask me this type of question – is that it’s clearly absurd. They themselves use Google all the time to find things. Millions of people each day clearly find information that’s useful. And yet, I’ve repeatedly seen notable tech journalists with a straight-face share how you can’t find anything on Google, that it is useless and so on. Google isn't completely useless, but it has became extremely painful to use. There was a time when Google returned exactly what I needed, or said that there's no result. For the last 5 years or so, I have to manually filter the results and, back when pagination still existed, I went directly to page 2 on some searches to skip the first dozen SEO-filled results. So, sure, I keep finding useful results on Google, but only after having to navigate through a sea of SEO-optimized links. And yes, I still keep using Google for a third of my search because I have no better choice on these topics.

drcongo 3y ago
Kagi. Better results than Google ever returned, and infinitely better than the GPT generated spam you get on the first several pages of Google these days.
- ipaddr 3y ago
  How does it rank nsfw content?
  - JohnFen 3y ago
    Give it a try. They have a free tier that is generous enough to find out if the service works well for you.
visarga 3y ago
> There was a time when Google returned exactly what I needed, or said that there's no result. For the last 5 years or so, I have to manually filter the results Sounds to me like Google search is also suffering from a lot of hallucinations, as if progress went backwards.

stillwithit 3y ago

The Verge is rent seeking; they need to be discoverable to exist. The Verge is like many other web assets these days; facing extinction and mad at everyone else over it. What a shock, past human social pageantry is being obsoleted. Thats never happened before; we all speak Latin still, and serve the British monarchy, don’t you know. Horse and buggy, rotary phone makers started a support group all the wannabe Lois Lanes and Clark Kent’s can join up with.

zenincognito 3y ago

I have been in this industry just 5 years shy of Danny. Danny was a big critic of Google back in the days. What he doesn't realise is that Google has deployed the oldest trick in the book. They took the most prominent critic in the industry and made him into their spokesman. Effectively, leveraging a lot of goodwill he holds in the industry into their own. I am not saying what Danny is saying is untrue or has malice. I am just saying that Google search has deteriorated immensely and is skewed to about 100 sites that dominate it competely. There is very little space for small business or genuine knowledgespeak unless you have a big budget to churn our content. Just look at Google's testimony in the recent antitrust case and you can see what Google says via the liasion team is opposite to how SEO actually works. Ex - Google using click data but constantly saying for over a decade they don't use the click data.

dannysullivan 3y ago
I get this view. But as my post explains, I'd quit writing about search. I was done. There wasn't going to be more criticism (or praise or whatever) from me because I'd retired from writing about search. I didn't have plans to go to Google when I retired. No one there even knew I was leaving. Which ... you or anyone can choose to believe or not, but that's how it is. I was far from the only critic (or advocate) for Google or other search engines. There are plenty of others. New people, and with good views, continue to come into the space. The idea of "Google hired me to quiet me," again, while I get it, just wouldn't resolve that. By the way, nor would some alternative idea that I somehow had secret details of spamming techniques make sense, either. Google had and still has an excellent spam team. They didn't need me to come in and somehow fill gaps. What Google gained by me coming in, I hope, is someone that both tries to help people better understand how the search engine works from within the search quality team (that's where I work, in that team) and also bring back into that team advocacy and feedback from the outside world (which typically, I realize, isn't that clear to those people outside Google -- here's an example I shared of this last week when asked: https://twitter.com/searchliaison/status/1720491595420856329 https://twitter.com/searchliaison/status/1720491595420856329 )

ChrisArchitect 3y ago

Related thread on the Verge article from earlier: https://news.ycombinator.com/item?id=38097938 https://news.ycombinator.com/item?id=38097938

micromacrofoot 3y ago

This is a case where I think both The Verge and Danny get things wrong, they're both far too biased. The Verge has come to some forgone conclusion about the decline of Google, and Danny works for Google.

Apreche 3y ago

> I almost certainly would not have said that everyone “who tried to rank higher by messing with the algorithm would be blocked.” That’s not a direct quote, but it also doesn’t make sense as a paraphrase. > We have ranking systems that largely do not “block” content. They look to reward good content and show it highly in results; lower-quality content drops. Even with actual spam, it’s unusual that we’d literally “block” an entire site. It just isn’t likely to rank well for general queries that aren’t somehow very specific to it. This, to me, is the fundamental problem that is ruining Google search results. There are many many sites that absolutely should be completely blocked. But Google will never block or downrank those sites because those sites are their golden goose. We've all done a search with a programming question, expecting the top result to either be the official documentation or Stack Overflow. Instead, we get a content farm site that has either stolen, or ML-generated, the answers from other sites that were actually responsible for crafting the content in the first place. You know the sites, I don't have to name them here. These content farm sites not only stole their content, but they are very unpleasant to visit. Sure, they may load fast. They may have the content you searched for. But they also have ads. Lots of ads. They also have ad blocker blockers. And they have little pop-ups that ask you to subscribe to the newsletter. Those sorts of things make a site unpleasant to visit. Those are the sorts of things that should make a site get heavily downranked, or completely blocked. No matter what content such a site has, a human doesn't want to suffer through the experience of going to that site. But Google can't do that. Those sites are their golden goose. Those sites are serving up ads powered by Google. If Google downranks sites that are a bad unpleasant ad-filled user experience, and upranks ad-free wonderful web sites, they're hosed. All those ad impressions their customers paid for will never happen if traffic to ad-laden web sites plummets. TL;DR: The ideal search engine would only allow a site to show up in a search result if that site was completely free of all ads, trackers, paywalls, pop-ups, or any other unpleasant similar thing. The entire rest of the web would be de-indexed. A search engine that itself is paid for by ads can never do that.

JohnFen 3y ago
> This, to me, is the fundamental problem that is ruining Google search results. That's certainly a large factor, but I think a greater factor (at least for me) is Google's use of ML to try to interpret what I "really" want when I write a search query instead of just going by what I actually wrote. Google makes terrible guesses as to my intent.

JohnMakin 3y ago

You will never, ever, ever, ever convince me that search is "better" now than it ever has been, or even more than a few years ago. This seems completely delusional to me. I define good search as me being able to quickly find what I am looking for. Approximately 1-2 years ago, I forget when it started, I noticed weird instances where no matter what I put into the engine and how long I scrolled or clicked around I absolutely could not find what I was looking for and gave up. I remember vividly the first time it happened, because it startled me. Google had never been like that. At first it was rare, then more often, these days it became so unusable I had to put it down. I think my final breaking point was trying to find the score of a sports game that had just happened a few hours before and being completely unable to. That's unimportant info, but since I frequently used google often in my day-to-day job, the inability to find what I need is a real serious issue. Bing has been great so far.

JohnMakin 3y ago
Good example and fun exercise for anyone to try - input to google search "Migrating off of GCP." You will only find documentation and videos how to migrate TO GCP. Yea, I don't buy that google mysteriously forgot how the english language works, sorry.
- cma 3y ago
  Google never worked well with that kind of language processing task in the heyday of its usefulness either. Searching for "dresses that aren't red" in image search returned almost 100% red dress images (maybe better now). Extremely common words like "off" and "of" were essentially ignored.
  - JohnMakin 3y ago
    You can try the same experiment with any combination of words, the results are basically the same.
  - johnnyanmac 3y ago
    I think the worse part is that there were tools around this and they also don't work anymore. Try to search "football -American" and you'll still end up with the search for American Football. search "football -nfl" and it will instead try to inject news of local college/high school teams instead of resorting to FIFA or other assossiations for the worldwide term. I never expected such boolean logic to be perfect, but it barely even seems to try anymore.

mediumsmart 3y ago

I don't know what the Verge is and I don't know much about SpamEngineOptimisation but I think that its not that complicated. You pay Google to have your rubbish rank higher and you pay Kagi to not have to look at that when searching for something useful. And there is still room for beautiful blogposts, interviews and discussions about the different angles and what the real issue is and how to look at this in a calm and objective way and what solutions might be on the horizon, hey, go for it. Choose a nice webfont and color palette and knock yourselves out. toodles.

nix0n 3y ago

> I do recall one question [...] along the lines of “So, what do you think about the idea that people can’t find things on Google Search,” to which I recall joking something like, “Yes, clearly no one finds things any more.” I really, actually, do not find things on Google anymore. The rate of success has gotten quite bad, under 50%, when it used to be closer to 95%. Sullivan says in his article that his role is "part of the search quality team", but he thinks this decline in quality is a laughing matter. It's not going to get better.

whstl 3y ago
I'm using ChatGPT for 90% of documentation search work, hallucinations and all, simply because Google has become THAT worse for development IMO. I can't find shit anymore. Sometimes when I have to use a specific library, I just download the documentation and search using VSCode or something.
- pastamania 3y ago
  I'm largely mystified at a lot of the 'Google is rubbish now' takes, but dev documentation is one of the areas that I genuinely find Google was better at a few years ago. I actually do tend to get answers to my questions, but a lot of them are desperately outdated and no longer relevant. My take on what happened is that Google has a metric in their algorithms called 'Query deserves freshness' - basically, identifying a search where the user isn't going to want old information, and artifically promoting more recent pages over better linked to but older documents. If someone wants to see the weather, they probably aren't looking for the weather on an extremely weird day 4 years ago just because a lot of people linked to it back then. They seem to have reduced that down, I suspect to reduce search being flooded with misinformation during the pandemic when all information was recent and the more controversial takes sometimes got more coverage. As an unexpected result, if I'm searching a dev query, it ends up surfacing a ton of results from years ago. Yeah, those pages may have more links and history and probably seem to an algorithm to be more trustworthy, but if it's for a library 20 versions old that long since stopped working the way the stackoverflow answer from 2011 says it did, that's pretty useless. Still, as a trade off, probably better me having to use the time filter dropdown to limit results to recent one's than someone necking bleach and horse de-wormer when they get Covid. And while I'm not quite as onboard with GPT being the death of search as some, dev documentation is one of those areas where generative AI is genuinely a better tool than web search ever was.

doogerdog 3y ago

The verge produces some good articles. This is not an example. I tried to read it, but quickly got bogged down in irrelevant fluff. So I skimmed the rest. Then I searched the article for "Sullivan". Every other occurrence is followed by, "is mad" "is pissed" "impatient corporate stooge", and other improbable attacks. Wow! She wrote a metric ton of words just to say, "I hate Danny". It is true that Google is a swamp. Thank goodness for Kagi.

lucubratory 3y ago

This whole article feels like Danny Sullivan cooling off and patiently trying to explain that no, we're all wrong, the sky really is red. It just doesn't fit with my experience. 60% of the time, for actual searches (not just using Google as my lazy equivalent of a memorised bookmarks bar because bookmarks don't work well on mobile) don't work, and I have to wade through crap that is mentally tiring to deal with just to be given the privilege of discovering that it didn't work, the information I want is not being provided by this search, or the toll being exacted by SEO spam crap for the information is soul destroying enough that I need to go somewhere else. As just one example, my husband likes to play Minecraft. Sometimes he asks me to look up something while he's playing. A couple of weeks ago he asked me several questions about a creature called a "Vex". This is what happened when I tried to use Google to answer those questions. One of the questions was about its health on Hard difficulty, another was if there were any blocks it couldn't phase through, another was if it's going to attack the villagers he's trying to keep safe in this little area. I don't want to search all of these individually and I figure all of these answers probably exist on one wiki page, so I search "Minecraft Vex wiki". The first result is a Fandom link; click on that, immediate banner scam ad covering half the screen, pop-up scam ad on the bottom, a third scam ad doing some weird scrolling thing on the side of the screen, that turns into an auto-playing video. One of the ads features a group of sexualised women selling deodorant in a tropical environment. The auto-playing video appears to be trying to tell me how great it would be to have a beer (I am 7 years sober, and I've told Google not to give me ads about alcohol). It has been approximately five seconds since I hit the search button on "Minecraft Vex wiki". It is a failure on Google's part that the first result is a site that has this experience. The second result is one of those IGN "wikis" that are generally worse than any other option and also full of ads, so I'm not interested but click to confirm - okay, this looks like it was written by a bad teenager, someone from Fiverr, or both. And it's absolutely full of ads, time to swipe back. The next thing in the results is four variations on questions about Vex, none of them being the questions I had. The next thing in the results is eight little thumbnails directly in the search results, with a little icon in the corner of each image indicating the website it's coming from - why would these image thumbnails ever, ever pop up on what is clearly a search for detailed, technical information? I didn't run an image search, I know how to click the image search button if I want that, and what I was looking for wasn't even possible to solve with an image thumbnail. They don't even have any text to tell me whether they're worth clicking on, they are a completely useless result for my search. What's worse, between them and the useless questions that I didn't ask, this takes up maybe five results worth of space. By the time I get to the third result on the page, I am maybe a third of the way down the page and it's the official Minecraft announcement page for the Vex mob. This is basically a good result for the Minecraft and Vex portions of the term (third time's the charm), but it still doesn't have the information I'm looking for, it's a general overview that doesn't answer any of the specific questions I had. Every other result from here on out was worse than this, until the result that was second to last. That was a minecraft.wiki link which wasn't torturous to navigate, and after clicking on it it was able to answer all of my questions, after I clarified some things with my husband. Why wasn't this result first? "Just make a good website" does not, in any way, describe what Fandom is providing, and Google recommends it to me as the first option. I don't know if it's specific to me because of my search history now, but conducting the same search now has the minecraft.wiki link as the first result, so it's clear Google is trying something to improve. I have zero confidence that the next time I search for information whose location I don't already know, that it will be any different. This is how at least half of my attempts to use Google to actually search go, and this was an absolute best case scenario for search because it was about probably the world's most popular video game with extensive technical documentation in existence, and I knew exactly what I wanted (a good wiki page on the Vex from Minecraft) and put it in the search, and because Google actually gave me the answer eventually instead of just crapping the bed and having nothing, like it often does. The entire experience was onerous and visually/auditorily offensive drudgery, so that I could answer someone's questions about a video game while they were focusing on another resource they were using to build something in the game. Does Google mark this as a "success" because I got the information I wanted? Because it wasn't successful, it made me want to gouge my eyes out just like every other time I use Google, and what it made me want most of all was an LLM from five years in the future that can just field those questions and provide answers without all of this... "experience". This was not my experience Googling questions 10 years ago. I do not know or care if this is all Danny Sullivan's fault, I don't know him from a can of bread, but it's someone's fault and frankly I hope the company that does this crap can't adapt fast enough and gets destroyed by AI. And you know what's funny? Before my first time using an LLM, I had just accepted that every time I was going to search on the internet it would be a little more soul destroying than it was before, every day for the rest of my life, and there was nothing I could ever do about it because no one would ever be able to compete with Google's monopoly. I had no anger because I had no hope that a solution to this was even possible. After my first time using an LLM with internet access, there were a boatload of problems and potential for abuse but I could immediately see that this technology could insulate me from the demons on the internet that make it a soul-trampling experience, and at least if it got the answers wrong it could say sorry instead of exposing me to unwanted pictures of a woman in a bikini covered in beer foam. Google is a bad, bad, bad experience, and the fact the SEO spammers and this Google employee think it's very effective is both unsurprising and unconvincing. Don't even get me started on the actual Google ads and their determination to make me read the Epoch Times, a newspaper run by a cult that my friend barely escaped from and which there's apparently no way to inform Google I am never going to click on so please don't show it to me anymore. Okay. I got mad but with reason, Google is rage-inducing in the modern era. Now I can move on with my day.