8 ms·

The Age of PageRank Is Over (2022)

76 points by dotcoma 2y ago

syndicatedjelly 2y ago
Amazingly prescient. The original version of that article was written in 2019
jaronilan 2y ago
Always found the concept of PR intriguing and the original paper to be “foundational”. I wrote an alternate reality fiction short story about PR and SEO. It is here: https://github.com/jaronilan/stories/blob/main/Duplicitous.pdf https://github.com/jaronilan/stories/blob/main/Duplicitous.p... Not the most “popular topic”, but thought people around HN might find it interesting.
geenat 2y ago
I'm not convinced Google has stopped using backlinks and other classic pagerank attributes for search ranking.
- stanislavb 2y ago
  I think you are right. They might be using a plethora of other factor as an addition to back links; however, genuine links from authoritative sources still seem to be the best "voting" mechanism to bubble up good content/results. I can imagine that Reddit could join the search war - first, they have so much user gen content that people are deliberately looking for; second, they could use the voting mechanisms that are already in place to give preference to the "better" content. Of course, I might be wrong :).
  - cryptoz 2y ago
    The part where you might be wrong is the part where Reddit does it. IIRC they have been “working really hard on a better search” for Reddit for…15 years? Every couple years spez will pop up and claim they almost have it! I wouldn’t hold my breath.
- 28304283409234 2y ago
  Nobody said they stopped. The article argues that the practise has become infinitely less effective, and Google is optimizing for their customers as opposed to their users.
- captainmuon 2y ago
  For many queries, it seems the top result is just Stackoverflow, Wikipedia or something simliar, then a couple of organic links, and then just junk. And on the second or third page, the results just stop. I have a feeling that at this point, it is a mixture of white- and blacklisting, and single-page classifiers based on heuristics or "AI". PageRank is surely in there, but probably not dominant. (Sidenote: I wonder if they secretly do ignore rel="nofollow" because so many links use it.) Funnily, I've used PageRank for some smaller search projects, and it still works very well if you use it in a vertical, say "educational resources" or "programming" or simliar. I just did a broad crawl, 3-4 steps from some known "good" seed URLs away, calculate PageRank and mix it in with Solr's classifiers.
28304283409234 2y ago
I will do you one better though, given my conversation with teenagers this weekend: Pagerank is dead, because webpages are dead. The only moment these kids use a search engine is when they do homework. In any and every other moment they just search "locally" in TikTok, or insta. It scares the shit out of me. (Edit: paying kagi customer here! Keep it up, kagi! I still love and need you!)
- Dalewyn 2y ago
  >because webpages are dead. I think it's more accurate to say that the internet and perhaps personal computing in general is dead. Practically everything about what makes a computer tick has been abstracted away, because rightfully or otherwise people today just don't care. When was the last time you saw someone actually using the address bar in a web browser instead of Googling? Or indeed Tictok'ing or Insta'ing. Nobody knows what a file or folder is either.
  - JumpCrisscross 2y ago
    > everything about what makes a computer tick has been abstracted away It’s always been abstractions. Even when we didn’t know it. (We mastered the technological use of transistors before we understood why they work.)
  - wiseowise 2y ago
    > When was the last time you saw someone actually using the address bar in a web browser instead of Googling? I see the guy in the mirror every once in a while, why?
    - audiodude 2y ago
      To be fair, I usually type the first few characters of the website I want to go to, and then pick the thing I want from the autocomplete results.
  - globular-toast 2y ago
    The majority of people never did personal computing. Remember the Eternal September where internet users were saddened by hordes of normies messing up the net? I feel like there was probably a spike around the early 2000s in number of people doing personal computing. Since then it's gone back to normal. Most people aren't interested and have no need for it. Tiktok etc is just the new TV: something to mindlessly rot away in front of. We also need to remember that even those people who did use computers never used them how we use them. They used Windows and Microsoft stuff. Hardly in control of their own computing. Part of me is saddened by it, but then I wonder if that isn't the case for everything: cyclists are sad about all the cars, cooks are sad about all the ready meals etc.
alex_suzuki 2y ago
(2022)
ricardo81 2y ago
I used to get sent a monthly cheque for $1K around the turn of the millennium simply to have a link on my home page, such was the power of Pagerank back then. This was handy as it allowed me to invest time into learning web development, plus the site was educational and paid for its existence. Producing content was harder back then and the web was a lot smaller. Google most definitely still use links to rank but are much more likely to discount/devalue links than they did historically.
- l5870uoo9y 2y ago
  Buying backlinks is still around and costly.
  - ricardo81 2y ago
    No doubt. Search still struggles to measure intent from the content creator PoV.
deleted 2y ago
[deleted]
dang 2y ago
Discussed at the time: The Age of PageRank Is Over - https://news.ycombinator.com/item?id=33537513 https://news.ycombinator.com/item?id=33537513 - Nov 2022 (373 comments)
Fordec 2y ago
Can the script on this then be flipped? Build a search engine, clearly smaller in scope and commercial utility, that if a site links to a payment or ad network, de-rank it heavily. Then the end result should be in theory, filled with what one would consider the "old" internet, primarily blogs and sites not trying to sell you things or abuse your data. None of the large companies would do it, but that would be the point.
- JumpCrisscross 2y ago
  Introducing: Kagi.
  - sph 2y ago
    That's not what Kagi does. Nor does its "Small Web" search mode, as it only searches blogs that have been manually added to a specific GitHub repo (so for most part is a collection of US tech blogs - not very diverse at all) No VC-backed or commercial search engine would do what OP is talking about. But I can see a use for a niche search engine that ranks websites inversely proportional to the number of trackers and ad networks they depend on. Heck, I would pay for that, but I'm a nerd. Maybe marginalia.nu would like this idea.
    - JumpCrisscross 2y ago
      > not what Kagi does Kagi maintains a "non-commercial index (Teclis) and non-commercial news index (TinyGem)" [1]. They also "prioritize non-commercial sources," implicitly downranking monetized sites. (Also, you can manually downrank problem domains [2].) My devices have randomly switched back to Google or DDG from time to time. The first thing I did was check my ad blocker was working--I was simply stunned by the amount of blogspam puke. > No VC-backed or commercial search engine would do what OP is talking about No ad based, i.e. free, engine can, not sustainably. A paid one obviously can and does. [1] https://help.kagi.com/kagi/search-details/search-quality.html https://help.kagi.com/kagi/search-details/search-quality.htm... [2] https://help.kagi.com/kagi/features/website-info-personalized-results.html https://help.kagi.com/kagi/features/website-info-personalize...
ben30 2y ago
When colleagues ask questions in Slack, I sometimes paste Kagi's search summary. Quick and usually spot-on. Funny thing is, I've told the team about Kagi, but not everyone's willing to pay/see the benefits yet. Meanwhile, I'm wondering why they're asking if a good search engine could answer it so easily.
trod123 2y ago
There are some subjects that are personal that no one will be comfortable searching for while its tied to a billing address (i.e. any login). Abortion in some states for example. It doesn't matter what any company says right now, the position is clear that the tune can and often does change the moment its inconvenient for them. Paying for the privilege of being targeted is crazy talk.
- carlosjobim 2y ago
  What if you don't care about privacy and only care for search quality? For example, when you're doing work related searches at work on your work device. Is that allowed?
- JumpCrisscross 2y ago
  > while its tied to a billing address (i.e. any login) Total false equivalence. Kagi accepts Bitcoin [1]. Logins are not identifiable in the way billing addresses are. > doesn't matter what any company says right now, the position is clear that the tune can and often does change the moment its inconvenient for them You think the free engines don't know who you are? [1] https://blog.kagi.com/accepting-paypal-bitcoin https://blog.kagi.com/accepting-paypal-bitcoin
  - mossTechnician 2y ago
    If you're a big fan of privacy and Kagi, then I'm not sure if you can appreciate the content of this article. After all, it describes their desire to build a search agent that knows everything about you, from your religion to your politics to your favorite brands.
  - trod123 2y ago
    bitcoin is traceable. not a false equivalence at all. Search history is kept tied to a profile/semaphore/data lake and then also tied financially to the way you pay, as well as the devices you use. When data is collected, it will be trolled through for anything and everything. Absence of data is just as unique. The only way is to blend into the masses generically. Free engines have a very hard time differentiating me from the mass of other web traffic. I compile my own browser to make it that way though, even my phone doesn't register an overnight location, and runs GrapheneOS. Importantly this should be a fundamental right of every person not just those that have the money and expertise to enforce it. The data collection shouldn't be allowed, its the equivalent of quartering a digital soldier in every home (something forbidden by the constitution). Any business funded and dependent on arbitrary preferential loans (from the printer, regardless of how indirect) is state-run and nationalized industry, and should be bound by the same constitutional requirements of government. I take my privacy seriously because of my job, and the indisputable fact that sensitive roles get targeted.
cryptos 2y ago
I like the idea to pay for quality content - or search results. If you don't pay for it, someone else will and the "sponsor" will probably have different objectives than you. So, you won't get the best information and it would be likely to waste money because of that (e.g. buying an advertised low-quality product).

76 points by dotcoma 2y ago

Amazingly prescient. The original version of that article was written in 2019

Always found the concept of PR intriguing and the original paper to be “foundational”. I wrote an alternate reality fiction short story about PR and SEO. It is here: https://github.com/jaronilan/stories/blob/main/Duplicitous.pdf https://github.com/jaronilan/stories/blob/main/Duplicitous.p... Not the most “popular topic”, but thought people around HN might find it interesting.

geenat 2y ago

I'm not convinced Google has stopped using backlinks and other classic pagerank attributes for search ranking.

stanislavb 2y ago
I think you are right. They might be using a plethora of other factor as an addition to back links; however, genuine links from authoritative sources still seem to be the best "voting" mechanism to bubble up good content/results. I can imagine that Reddit could join the search war - first, they have so much user gen content that people are deliberately looking for; second, they could use the voting mechanisms that are already in place to give preference to the "better" content. Of course, I might be wrong :).
- cryptoz 2y ago
  The part where you might be wrong is the part where Reddit does it. IIRC they have been “working really hard on a better search” for Reddit for…15 years? Every couple years spez will pop up and claim they almost have it! I wouldn’t hold my breath.
28304283409234 2y ago
Nobody said they stopped. The article argues that the practise has become infinitely less effective, and Google is optimizing for their customers as opposed to their users.
captainmuon 2y ago
For many queries, it seems the top result is just Stackoverflow, Wikipedia or something simliar, then a couple of organic links, and then just junk. And on the second or third page, the results just stop. I have a feeling that at this point, it is a mixture of white- and blacklisting, and single-page classifiers based on heuristics or "AI". PageRank is surely in there, but probably not dominant. (Sidenote: I wonder if they secretly do ignore rel="nofollow" because so many links use it.) Funnily, I've used PageRank for some smaller search projects, and it still works very well if you use it in a vertical, say "educational resources" or "programming" or simliar. I just did a broad crawl, 3-4 steps from some known "good" seed URLs away, calculate PageRank and mix it in with Solr's classifiers.

28304283409234 2y ago

I will do you one better though, given my conversation with teenagers this weekend: Pagerank is dead, because webpages are dead. The only moment these kids use a search engine is when they do homework. In any and every other moment they just search "locally" in TikTok, or insta. It scares the shit out of me. (Edit: paying kagi customer here! Keep it up, kagi! I still love and need you!)

Dalewyn 2y ago
>because webpages are dead. I think it's more accurate to say that the internet and perhaps personal computing in general is dead. Practically everything about what makes a computer tick has been abstracted away, because rightfully or otherwise people today just don't care. When was the last time you saw someone actually using the address bar in a web browser instead of Googling? Or indeed Tictok'ing or Insta'ing. Nobody knows what a file or folder is either.
- JumpCrisscross 2y ago
  > everything about what makes a computer tick has been abstracted away It’s always been abstractions. Even when we didn’t know it. (We mastered the technological use of transistors before we understood why they work.)
- wiseowise 2y ago
  > When was the last time you saw someone actually using the address bar in a web browser instead of Googling? I see the guy in the mirror every once in a while, why?
  - audiodude 2y ago
    To be fair, I usually type the first few characters of the website I want to go to, and then pick the thing I want from the autocomplete results.
- globular-toast 2y ago
  The majority of people never did personal computing. Remember the Eternal September where internet users were saddened by hordes of normies messing up the net? I feel like there was probably a spike around the early 2000s in number of people doing personal computing. Since then it's gone back to normal. Most people aren't interested and have no need for it. Tiktok etc is just the new TV: something to mindlessly rot away in front of. We also need to remember that even those people who did use computers never used them how we use them. They used Windows and Microsoft stuff. Hardly in control of their own computing. Part of me is saddened by it, but then I wonder if that isn't the case for everything: cyclists are sad about all the cars, cooks are sad about all the ready meals etc.

alex_suzuki 2y ago

(2022)

ricardo81 2y ago

I used to get sent a monthly cheque for $1K around the turn of the millennium simply to have a link on my home page, such was the power of Pagerank back then. This was handy as it allowed me to invest time into learning web development, plus the site was educational and paid for its existence. Producing content was harder back then and the web was a lot smaller. Google most definitely still use links to rank but are much more likely to discount/devalue links than they did historically.

l5870uoo9y 2y ago
Buying backlinks is still around and costly.
- ricardo81 2y ago
  No doubt. Search still struggles to measure intent from the content creator PoV.

deleted 2y ago

[deleted]

dang 2y ago

Discussed at the time: The Age of PageRank Is Over - https://news.ycombinator.com/item?id=33537513 https://news.ycombinator.com/item?id=33537513 - Nov 2022 (373 comments)

Fordec 2y ago

Can the script on this then be flipped? Build a search engine, clearly smaller in scope and commercial utility, that if a site links to a payment or ad network, de-rank it heavily. Then the end result should be in theory, filled with what one would consider the "old" internet, primarily blogs and sites not trying to sell you things or abuse your data. None of the large companies would do it, but that would be the point.

JumpCrisscross 2y ago
Introducing: Kagi.
- sph 2y ago
  That's not what Kagi does. Nor does its "Small Web" search mode, as it only searches blogs that have been manually added to a specific GitHub repo (so for most part is a collection of US tech blogs - not very diverse at all) No VC-backed or commercial search engine would do what OP is talking about. But I can see a use for a niche search engine that ranks websites inversely proportional to the number of trackers and ad networks they depend on. Heck, I would pay for that, but I'm a nerd. Maybe marginalia.nu would like this idea.
  - JumpCrisscross 2y ago
    > not what Kagi does Kagi maintains a "non-commercial index (Teclis) and non-commercial news index (TinyGem)" [1]. They also "prioritize non-commercial sources," implicitly downranking monetized sites. (Also, you can manually downrank problem domains [2].) My devices have randomly switched back to Google or DDG from time to time. The first thing I did was check my ad blocker was working--I was simply stunned by the amount of blogspam puke. > No VC-backed or commercial search engine would do what OP is talking about No ad based, i.e. free, engine can, not sustainably. A paid one obviously can and does. [1] https://help.kagi.com/kagi/search-details/search-quality.html https://help.kagi.com/kagi/search-details/search-quality.htm... [2] https://help.kagi.com/kagi/features/website-info-personalized-results.html https://help.kagi.com/kagi/features/website-info-personalize...

ben30 2y ago

When colleagues ask questions in Slack, I sometimes paste Kagi's search summary. Quick and usually spot-on. Funny thing is, I've told the team about Kagi, but not everyone's willing to pay/see the benefits yet. Meanwhile, I'm wondering why they're asking if a good search engine could answer it so easily.

trod123 2y ago

There are some subjects that are personal that no one will be comfortable searching for while its tied to a billing address (i.e. any login). Abortion in some states for example. It doesn't matter what any company says right now, the position is clear that the tune can and often does change the moment its inconvenient for them. Paying for the privilege of being targeted is crazy talk.

carlosjobim 2y ago
What if you don't care about privacy and only care for search quality? For example, when you're doing work related searches at work on your work device. Is that allowed?
JumpCrisscross 2y ago
> while its tied to a billing address (i.e. any login) Total false equivalence. Kagi accepts Bitcoin [1]. Logins are not identifiable in the way billing addresses are. > doesn't matter what any company says right now, the position is clear that the tune can and often does change the moment its inconvenient for them You think the free engines don't know who you are? [1] https://blog.kagi.com/accepting-paypal-bitcoin https://blog.kagi.com/accepting-paypal-bitcoin
- mossTechnician 2y ago
  If you're a big fan of privacy and Kagi, then I'm not sure if you can appreciate the content of this article. After all, it describes their desire to build a search agent that knows everything about you, from your religion to your politics to your favorite brands.
- trod123 2y ago
  bitcoin is traceable. not a false equivalence at all. Search history is kept tied to a profile/semaphore/data lake and then also tied financially to the way you pay, as well as the devices you use. When data is collected, it will be trolled through for anything and everything. Absence of data is just as unique. The only way is to blend into the masses generically. Free engines have a very hard time differentiating me from the mass of other web traffic. I compile my own browser to make it that way though, even my phone doesn't register an overnight location, and runs GrapheneOS. Importantly this should be a fundamental right of every person not just those that have the money and expertise to enforce it. The data collection shouldn't be allowed, its the equivalent of quartering a digital soldier in every home (something forbidden by the constitution). Any business funded and dependent on arbitrary preferential loans (from the printer, regardless of how indirect) is state-run and nationalized industry, and should be bound by the same constitutional requirements of government. I take my privacy seriously because of my job, and the indisputable fact that sensitive roles get targeted.

cryptos 2y ago

I like the idea to pay for quality content - or search results. If you don't pay for it, someone else will and the "sponsor" will probably have different objectives than you. So, you won't get the best information and it would be likely to waste money because of that (e.g. buying an advertised low-quality product).