5 ms·

PageRank explained

148 points by pkoird 1mo ago

jmkd 1mo ago
One is reminded of Damien Hirst's famed retort to a critic who said "Well I could have pickled a shark" ... "But you didn't, did you. I did."
- jaggederest 1mo ago
  Reminds me of Teddy Roosevelt's "Citizenship in a Republic" speech, where he talked about the man in the arena: > It is not the critic who counts; not the man who points out how the strong man stumbles or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, and comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows the great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who know neither victory nor defeat. https://www.presidency.ucsb.edu/documents/address-the-sorbonne-paris-france-citizenship-republic https://www.presidency.ucsb.edu/documents/address-the-sorbon...
  - otterley 1mo ago
    My favorite related pithy quote is "criticism is a minimum-wage job."
smcg 1mo ago
Well, I was a child in 1996, so probably not. Tying relevancy to link frequency was definitely a novel idea at the time, even if it seems "obvious" or simple in retrospect.
inigyou 1mo ago
Importantly, PageRank doesn't work today - you need something else. It was one of many possible ranking hacks, and one that worked at the particular time in that particular state of the web where nobody was gaming links because PageRank didn't exist yet. Maybe you could invent a good ranking algorithm for the modern internet, perhaps just the reciprocal of the number of ads on the page, minus its AI detector score, but probably not that. Unimportantly, it's named after Larry Page, not after the fact that it ranks pages.
- MoltenMan 1mo ago
  Another Goodhart's Law casualty...
- montag 1mo ago
  Really?? And not BrinRank?
  - esprehn 1mo ago
    Larry designed the algorithm and Brin implemented it. Not so different than what we saw with transformers and LLMs. The people who came up wit the ideas and the algorithms were joined by others who implemented them. Together they made it all work.
    - utopcell 1mo ago
      There's no credible reference to this. The most likely scenario is that they worked together in designing the system. They also credited their professors, Motwani and Winograd, for it.
  - inigyou 1mo ago
    Perhaps the second half of the word referred to Brin, after his smell after coding it for a week nonstop. Only a silly hypothesis of course.
- jjice 1mo ago
  > Unimportantly, it's named after Larry Page, not after the fact that it ranks pages. I've always taken it as a happy double play since it is named after Page, but also ranks pages.
ianbooker 1mo ago
PageRank is fascinating, since it is so easy to explain. Yet, this is not even half the work. It like a third of the way. Before you could have invented PageRank, you must think in graphs. That is possible in 1996, but not as widespread as today. After you invented PageRank, you still need to deploy it. Again, possible but challenging as well. Is Python performant enough in 96? Can you afford more than 4MB RAM? At least Lego will not sue you for using their bricks to build a server rack in 1996.
- Retr0id 1mo ago
  > you must think in graphs. That is possible in 1996, but not as widespread as today. Could you elaborate? I'm a very graph-oriented thinker, and I was never aware this was some kind of declining skill (For context I was not alive in '96).
  - seanw444 1mo ago
    Plus a lot of the advanced graph-related algorithms were discovered by the 80s.
    - tantalor 1mo ago
      Yeah, this was all done in Bell Labs starting in the 1960s
  - packetlost 1mo ago
    I think you're misreading it (I did the first time as well). GP is saying "graph thinking" was not as common in 1996 as it is today, so is saying the opposite of it being a declining skill.
    - Retr0id 1mo ago
      Ah, yeah, that'd do it.
- ssivark 1mo ago
  > Before you could have invented PageRank, you must think in graphs. For anyone puzzling over what is the graph-based perspective, there's actually a very elegant and simple mathematical way to derive page rank. Define the directed graph of web links / citations through its adjacency matrix: web pages as nodes and links as directed edges. Represent a web surfer as a random walker on this graph, and run forth (simulate) the probability distribution of where they might end up. If you've studied linear algebra you would see an immediate analogy between the PageRank algorithm how the power method is used to find the dominant eigenvector (of the transition matrix, which is a normalized version of the adjacency matrix). This process is basically equivalent to implementing the diffusion process generated by the discrete graph laplacian. And simulating the random walk to find the long term stationary probability dstribution over nodes is akin to finding the zero eigenvector of this graph laplacian -- because it must generate "zero change" on the fixed point state.
salemh 1mo ago
[dead]
lproven 1mo ago
Yup. I was there, using the web, back then. I used Altavista dozens of times a day. It wasn't very good. I never thought of it. I never thought of the Million Dollar Homepage, either. It's still there! https://milliondollarhomepage.com/ https://milliondollarhomepage.com/ I remember the first ever HTML CV (résumé) being published. I am Slashdot user #6030. I used it for ages before I created a user account. I was already paying for my own personal email address in 1991 when timbl revealed the WWW to the world. I thought it was a gimmick. It'd never catch on. We already had Gopher and Archie and Veronica. deep sigh
- mmargenot 1mo ago
  In recent news this page that's been somewhat viral recently seems an interesting successor to the million dollar homepage: https://outbid.lol/ https://outbid.lol/ A little nihilistic, but I find it charming.
  - basilikum 1mo ago
    Milliondollarhomepage is a little strange and irritating, you're just buying pixels after all. But it is interesting and has a certain conflicting charm to it. Outbid is just sad. People pay money to get on a slop board?
    - mmargenot 1mo ago
      basically, yeah. wild flash in the pan, but i think it's cute in that it's signaling reduced down to the absolute bare bones.
lindig 1mo ago
How do you compute page rank for billion of pages that have cyclic links? Is that not the problem right after the initial idea?
- jmalicki 1mo ago
  That is not at all a problem. Links dampen their effect and you run it until convergence - read the paper. It's no different than summing an infinite convergent series, $\sum_{i=0}^n a^{-n}$. Also, WTF, it seems impossible to find a PDF of the original paper still on the web without a paywall.
  - hluska 1mo ago
    You’re underestimating the difficulty - actually downloading, parsing and updating the system with billions of pages was a very hard thing in the 1990s.
    - icedchai 1mo ago
      Good thing there weren't billions of pages in the 90's.
    - jmalicki 1mo ago
      It's not an algorithmic issue, though, that's a systems issue. GGP was talking about the cyclic link nature which isn't an issue of size, it's an issue of convergence.
    - OroPla 1mo ago
      In 1997, there were only a hundred thousand web pages. Not even millions. The one billion mark would not be breached until 2016 according to the results of a quick search.
truthbe 1mo ago
If I had received funding from DARPA, and NASA, perhaps I could have.
- rco8786 1mo ago
  Pretty sure that came after. And funding never made anyone smarter.
  - truthbe 1mo ago
    Funding didn’t make them smarter, but it did buy the hardware to scale it. Other guys had the exact same idea at the exact same time like RankDex and IBM's HITS algorithm. Google just got the cash first to actually build the server farms to run it.
  - CPLX 1mo ago
    > funding never made anyone smarter One of my favorite things about HN is the way that you can find someone who has produced definitive proof of the non-existence of the public education system in an offhand comment.
    - rco8786 1mo ago
      Please, tell me more about my educational background.
  - SJC_Hacker 1mo ago
    They used Stanfords existing infrastructure … I don’t think the grant they were on paid for it directly. Pretty sure the cluster was a resource shared by several labs
    - deleted 1mo ago
      [deleted]
  - shirol 1mo ago
    But now, you can hire smart people.
CalChris 1mo ago
> Sergey Brin and Larry Page came up with this precise algorithm, i.e., PageRank, which was one of the key algorithms that helped catapult Google into a household name and made them tons of money. Both Sergey and Larry were grad students at Stanford, so their coming up with such an amazing algorithm doesn’t seem surprising. No, Brin wasn’t a co-inventor of PageRank. https://patents.google.com/patent/US7058628B1/en https://patents.google.com/patent/US7058628B1/en
- Kranar 1mo ago
  Inventor in the context of a U.S. patent has a very narrow and technical meaning, different from the broader concept of "coming up with something." Larry Page and Sergey Brin developed PageRank together as grad students in Stanford's Digital Library Project, led by Hector Garcia-Molina. This is simply undisputable and both are listed as co-authors of the PageRank paper [1]. The list of inventors on a patent, especially nowadays, should not be seen as a historical finding about credit. [1] https://www.semanticscholar.org/paper/The-PageRank-Citation-Ranking-%3A-Bringing-Order-to-Page-Brin/eb82d3035849cd23578096462ba419b53198a556 https://www.semanticscholar.org/paper/The-PageRank-Citation-...
  - pcrh 1mo ago
    Inventorship of patents has a clear legal basis, and is quite different from the basis of scientific authorship. The system disincentifies adding "ghost" inventors since incorrect listing of the inventors can invalidate a patent. So, yes, it's likely that the patent attorneys hired by Stanford advised that only Page should be listed as an inventor. Stanford gets its money via the assignment, not the inventor list.
  - srean 1mo ago
    If I recall correctly, it was for course project for the class taught by late Rajeev Motwani. Not contradicting your comment, adding more context.
paxys 1mo ago
Thinking of an algorithm in the abstract is one thing, implementing it at scale is another. Yes you could have invented PageRank, but could you also have invented MapReduce, BigFiles/Google File System (GFS), Google Web Server, Bigtable, Protobuf? Then spun up fault-tolerant clusters consisting of cheap commodity PC hardware in an era where AWS wasn't even an idea yet? Then invented the concept of Borg to manage this hardware globally?
- blueg3 1mo ago
  When Google started, Beowulf clusters were all the rage. So it wouldn't be a big conceptual leap at all. But productionizing it is serious work.
  - cmrdporcupine 1mo ago
    There were smart people around thinking about distributed systems to hire in 1996 just as there is now. It's a dream job for nerds. Productionizing it is not at all impossible if you have money to pay people to do it. It's not like there weren't examples of real world distributed job systems out there already (just different). Money you get by being a pair of connected Stanford grads living in the most tech connected place in the world, with Stanford alumni and venture capital connected people all around you. The CS is part is medium-hard. The productionizing part is a hiring problem. Getting the capital is a whole other aspect. I knew lots of smart people in 1996 in the first .com wave. None of us made any money :-) I should have moved to San Francisco in 97 like I was originally planning. Oops. (And no, I could not have done what Jeff Dean & Sanjay did but I would have loved to have tried ;-) )
deleted 1mo ago
[deleted]
ctbeiser 1mo ago
I could not have invented it. I was one year old.
nayuki 1mo ago
Here are two excellent videos that explain and visualize the PageRank algorithm: * [2020-06-17] Spanning Tree - "How Google's PageRank Algorithm Works" (5m16s): https://www.youtube.com/watch?v=meonLcN7LD4 https://www.youtube.com/watch?v=meonLcN7LD4 * [2022-05-23] Reducible - "PageRank: A Trillion Dollar Algorithm" (25m25s): https://www.youtube.com/watch?v=JGQe4kiPnrU https://www.youtube.com/watch?v=JGQe4kiPnrU
saltysalt 1mo ago
Let's not forget that PageRank unintentionally spawned the SEO industry.
- targetweb 1mo ago
  True
- srean 1mo ago
  The blame should not rest at Pagerank's feet. The moment the web became a place to make money SEO would have cropped up to target any other ranking system, had that become the defacto standard. In fact the ranking system was not raw Pagerank's, it was one of the many signals used to rank a page. It was the resulting ranking that SEO targets, not Pagerank specifically.
- stefs 26d ago
  i don't think that's correct. SEO has been popular long before google. many websites were overloaded with long lists of unrelated keywords at the bottom. google became the most popular search engine exactly because those SEO keyword spam lists didn't work anymore and, after a while, they vanished. blogs, on the other hand, worked exceptionally well, because they used to link to other blogs' entries a lot, and blogs also used to be (comparably) high quality content. so the main SEO guideline for getting a good page rank used to be: have a blog/news and publish lots of high quality content that bloggers link to.
notaigenerated 1mo ago
You could also have invented a wheel because it's so obvious.
- kqr 1mo ago
  The problem with the wheel is not inventing it so much as finding ways to use it. Aside from capstans and pulleys, the wheel is mostly only useful in combination with good infrastructure or complicated machinery.
seebeegeebee 1mo ago
Step 1: be sponsored by the CIA
- DANmode 1mo ago
  https://qz.com/1145669/googles-true-origin-partly-lies-in-cia-and-nsa-research-grants-for-mass-surveillance https://qz.com/1145669/googles-true-origin-partly-lies-in-ci... More by this author: > a new crowd-funded investigative journalism project, breaks the exclusive story of how the United States intelligence community funded, nurtured and incubated Google as part of a drive to dominate the world through control of information. Seed-funded by the NSA and CIA, Google was merely the first among a plethora of private sector start-ups co-opted by US intelligence to retain ‘information superiority.’ > The origins of this ingenious strategy trace back to a secret Pentagon-sponsored group, that for the last two decades has functioned as a bridge between the US government and elites across the business, industry, finance, corporate, and media sectors. The group has allowed some of the most powerful special interests in corporate America to systematically circumvent democratic accountability and the rule of law to influence government policies, as well as public opinion in the US and around the world. The results have been catastrophic: NSA mass surveillance, a permanent state of global war, and a new initiative to transform the US military into Skynet. https://web.archive.org/web/20190722034225/https://medium.com/insurge-intelligence/how-the-cia-made-google-e836451a959e https://web.archive.org/web/20190722034225/https://medium.co...
joe_the_user 1mo ago
it’ll give you an article on “Hotels for Chickens” if you search “Hotels” because the word matches What if I fricken want fricken "Hotels for Chickens", what about that, huh? I mean, page rank was useful but it was also a step in the direction of "search give you what it think you, not what you asked for" and I think now we can how far and dubious progress in that direction has been.
yungookim 1mo ago
curious, how do you decide on which page to start from? Popular directories?
snarfy 1mo ago
Yes anybody can be in the right place at the right time.
jefflinwood 1mo ago
I actually built this, and shipped it, in 1996, with no knowledge of page rank, citation analysis, or bibliometrics, for an internal/external search engine for the Envirolink web site. Envirolink was a directory of environmental web sites, so they already had a list of URLs to crawl. The reason it was feasible to build was that it was a fairly constrained list of URLs, it wasn't the entire web. I didn't really know what I was doing (I was 17), but it was an awesome unpaid summer internship. There were two parts of the search engine - a crawler and the search engine. Both were written in Perl.
steve1977 1mo ago
No mention of RankDex?
mpalmer 1mo ago
All credit to you for writing this yourself! PageRank, at its core, symbolizes these basic properties. This isn't what "symbolizes" means. You could use "embodies", "satisfies", "models", etc.
yu3zhou4 1mo ago
Also PageRank as a Markov chain [0] https://math.libretexts.org/Bookshelves/Linear_Algebra/Understanding_Linear_Algebra_(Austin)/04%3A_Eigenvalues_and_eigenvectors/4.05%3A_Markov_chains_and_Google%27s_PageRank_algorithm https://math.libretexts.org/Bookshelves/Linear_Algebra/Under...
Animats 1mo ago
As, I think, Page points out in the patent, PageRank's idea comes from Science Citation Index. That was an inverted list of scientific references, where you could look up an scientific paper in an expensive set of bound books and find all the papers in which it was later referenced. You can then use this to see who's getting referenced a lot, which is an ego trip in academia. Academic libraries had copies of that index. Now everybody has that kind of info, but when it had to be done by hand, it was hard. Inverting the huge, sparse matrix of references for PageRank was expensive. Originally, Google did it about once a week. The big breakthrough was when someone (who?) figured out how to do it incrementally at scale.
BiraIgnacio 1mo ago
Related, fascinating foundational work Citation index https://en.wikipedia.org/wiki/Citation_index https://en.wikipedia.org/wiki/Citation_index Shepard's Citations https://en.wikipedia.org/wiki/Shepard's_Citations https://en.wikipedia.org/wiki/Shepard's_Citations
kwanbix 1mo ago
What I always wonder is, how does google knows how to go to abc.com, xyz.com, mypage.com, etc. so it can crawl them?
- SJC_Hacker 1mo ago
  It starts with seed URLs and fans out from there Back in 1996, this could be done by hand
  - kwanbix 1mo ago
    thanks! I imagined it will be something like that. Maybe Google used yahoo index back then as seed?
    - orsenthil 1mo ago
      Yes. I think, there used be a directory of a links published, like an index.
TimCTRL 1mo ago
It's giving "Attention is all you need" but in the year 1996
williamcotton 1mo ago
Here's an in-depth explanation with an interactive demo that I made many moons ago: https://web.archive.org/web/20130728183938/williamcotton.com/pagerank-explained-with-javascript https://web.archive.org/web/20130728183938/williamcotton.com...
kerblang 1mo ago
PageRank is really the beginning of popularity as proxy for authentic/truthful/accurate/etc. Then we move on to upvotes, retweets, likes, listens, views, please-please-please-subscribe-to-my-channel, reviews etc. Thus the continuum of the "social" internet. We tried really hard to make truth into a capitalist endeavor.
1vuio0pswjnm7 1mo ago
Original HN title: "You could have invented PageRank"
1vuio0pswjnm7 1mo ago
Popularity is not equal to relevance Except in the case of advertising Even if, hypothetically, company knows what user is searching for before user finishes typing a query, it does not mean company is going to deliver it to user Company is paid by advertisers, not users Company can deliver something _popular_, call it "relevant", make bank "Popular" grows audience, good for advertisers "Relevant" not necessarily good for advertisers, only good for user Company serves its customers: advertisers Company does not need PageRank to determine popularity, only search traffic (search query data) (Of course a result could be both relevant and popular. But a result could also be relevant and _unpopular_. Company is promotes the former to the exclusion of the later. It profits by selling advertising services to advertisers, not information services to www users)
- 1vuio0pswjnm7 1mo ago
  To stay on top, the company buys search traffic For example, paying vendors to be the "default" search engine

148 points by pkoird 1mo ago

jmkd 1mo ago

One is reminded of Damien Hirst's famed retort to a critic who said "Well I could have pickled a shark" ... "But you didn't, did you. I did."

jaggederest 1mo ago
Reminds me of Teddy Roosevelt's "Citizenship in a Republic" speech, where he talked about the man in the arena: > It is not the critic who counts; not the man who points out how the strong man stumbles or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, and comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows the great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who know neither victory nor defeat. https://www.presidency.ucsb.edu/documents/address-the-sorbonne-paris-france-citizenship-republic https://www.presidency.ucsb.edu/documents/address-the-sorbon...
- otterley 1mo ago
  My favorite related pithy quote is "criticism is a minimum-wage job."

smcg 1mo ago

Well, I was a child in 1996, so probably not. Tying relevancy to link frequency was definitely a novel idea at the time, even if it seems "obvious" or simple in retrospect.

inigyou 1mo ago

Importantly, PageRank doesn't work today - you need something else. It was one of many possible ranking hacks, and one that worked at the particular time in that particular state of the web where nobody was gaming links because PageRank didn't exist yet. Maybe you could invent a good ranking algorithm for the modern internet, perhaps just the reciprocal of the number of ads on the page, minus its AI detector score, but probably not that. Unimportantly, it's named after Larry Page, not after the fact that it ranks pages.

MoltenMan 1mo ago
Another Goodhart's Law casualty...
montag 1mo ago
Really?? And not BrinRank?
- esprehn 1mo ago
  Larry designed the algorithm and Brin implemented it. Not so different than what we saw with transformers and LLMs. The people who came up wit the ideas and the algorithms were joined by others who implemented them. Together they made it all work.
  - utopcell 1mo ago
    There's no credible reference to this. The most likely scenario is that they worked together in designing the system. They also credited their professors, Motwani and Winograd, for it.
- inigyou 1mo ago
  Perhaps the second half of the word referred to Brin, after his smell after coding it for a week nonstop. Only a silly hypothesis of course.
jjice 1mo ago
> Unimportantly, it's named after Larry Page, not after the fact that it ranks pages. I've always taken it as a happy double play since it is named after Page, but also ranks pages.

ianbooker 1mo ago

PageRank is fascinating, since it is so easy to explain. Yet, this is not even half the work. It like a third of the way. Before you could have invented PageRank, you must think in graphs. That is possible in 1996, but not as widespread as today. After you invented PageRank, you still need to deploy it. Again, possible but challenging as well. Is Python performant enough in 96? Can you afford more than 4MB RAM? At least Lego will not sue you for using their bricks to build a server rack in 1996.

Retr0id 1mo ago
> you must think in graphs. That is possible in 1996, but not as widespread as today. Could you elaborate? I'm a very graph-oriented thinker, and I was never aware this was some kind of declining skill (For context I was not alive in '96).
- seanw444 1mo ago
  Plus a lot of the advanced graph-related algorithms were discovered by the 80s.
  - tantalor 1mo ago
    Yeah, this was all done in Bell Labs starting in the 1960s
- packetlost 1mo ago
  I think you're misreading it (I did the first time as well). GP is saying "graph thinking" was not as common in 1996 as it is today, so is saying the opposite of it being a declining skill.
  - Retr0id 1mo ago
    Ah, yeah, that'd do it.
ssivark 1mo ago
> Before you could have invented PageRank, you must think in graphs. For anyone puzzling over what is the graph-based perspective, there's actually a very elegant and simple mathematical way to derive page rank. Define the directed graph of web links / citations through its adjacency matrix: web pages as nodes and links as directed edges. Represent a web surfer as a random walker on this graph, and run forth (simulate) the probability distribution of where they might end up. If you've studied linear algebra you would see an immediate analogy between the PageRank algorithm how the power method is used to find the dominant eigenvector (of the transition matrix, which is a normalized version of the adjacency matrix). This process is basically equivalent to implementing the diffusion process generated by the discrete graph laplacian. And simulating the random walk to find the long term stationary probability dstribution over nodes is akin to finding the zero eigenvector of this graph laplacian -- because it must generate "zero change" on the fixed point state.

salemh 1mo ago

[dead]

lproven 1mo ago

Yup. I was there, using the web, back then. I used Altavista dozens of times a day. It wasn't very good. I never thought of it. I never thought of the Million Dollar Homepage, either. It's still there! https://milliondollarhomepage.com/ https://milliondollarhomepage.com/ I remember the first ever HTML CV (résumé) being published. I am Slashdot user #6030. I used it for ages before I created a user account. I was already paying for my own personal email address in 1991 when timbl revealed the WWW to the world. I thought it was a gimmick. It'd never catch on. We already had Gopher and Archie and Veronica. deep sigh

mmargenot 1mo ago
In recent news this page that's been somewhat viral recently seems an interesting successor to the million dollar homepage: https://outbid.lol/ https://outbid.lol/ A little nihilistic, but I find it charming.
- basilikum 1mo ago
  Milliondollarhomepage is a little strange and irritating, you're just buying pixels after all. But it is interesting and has a certain conflicting charm to it. Outbid is just sad. People pay money to get on a slop board?
  - mmargenot 1mo ago
    basically, yeah. wild flash in the pan, but i think it's cute in that it's signaling reduced down to the absolute bare bones.

lindig 1mo ago

How do you compute page rank for billion of pages that have cyclic links? Is that not the problem right after the initial idea?

jmalicki 1mo ago
That is not at all a problem. Links dampen their effect and you run it until convergence - read the paper. It's no different than summing an infinite convergent series, $\sum_{i=0}^n a^{-n}$. Also, WTF, it seems impossible to find a PDF of the original paper still on the web without a paywall.
- hluska 1mo ago
  You’re underestimating the difficulty - actually downloading, parsing and updating the system with billions of pages was a very hard thing in the 1990s.
  - icedchai 1mo ago
    Good thing there weren't billions of pages in the 90's.
  - jmalicki 1mo ago
    It's not an algorithmic issue, though, that's a systems issue. GGP was talking about the cyclic link nature which isn't an issue of size, it's an issue of convergence.
  - OroPla 1mo ago
    In 1997, there were only a hundred thousand web pages. Not even millions. The one billion mark would not be breached until 2016 according to the results of a quick search.

truthbe 1mo ago

If I had received funding from DARPA, and NASA, perhaps I could have.

rco8786 1mo ago
Pretty sure that came after. And funding never made anyone smarter.
- truthbe 1mo ago
  Funding didn’t make them smarter, but it did buy the hardware to scale it. Other guys had the exact same idea at the exact same time like RankDex and IBM's HITS algorithm. Google just got the cash first to actually build the server farms to run it.
- CPLX 1mo ago
  > funding never made anyone smarter One of my favorite things about HN is the way that you can find someone who has produced definitive proof of the non-existence of the public education system in an offhand comment.
  - rco8786 1mo ago
    Please, tell me more about my educational background.
- SJC_Hacker 1mo ago
  They used Stanfords existing infrastructure … I don’t think the grant they were on paid for it directly. Pretty sure the cluster was a resource shared by several labs
  - deleted 1mo ago
    [deleted]
- shirol 1mo ago
  But now, you can hire smart people.

CalChris 1mo ago

> Sergey Brin and Larry Page came up with this precise algorithm, i.e., PageRank, which was one of the key algorithms that helped catapult Google into a household name and made them tons of money. Both Sergey and Larry were grad students at Stanford, so their coming up with such an amazing algorithm doesn’t seem surprising. No, Brin wasn’t a co-inventor of PageRank. https://patents.google.com/patent/US7058628B1/en https://patents.google.com/patent/US7058628B1/en

Kranar 1mo ago
Inventor in the context of a U.S. patent has a very narrow and technical meaning, different from the broader concept of "coming up with something." Larry Page and Sergey Brin developed PageRank together as grad students in Stanford's Digital Library Project, led by Hector Garcia-Molina. This is simply undisputable and both are listed as co-authors of the PageRank paper [1]. The list of inventors on a patent, especially nowadays, should not be seen as a historical finding about credit. [1] https://www.semanticscholar.org/paper/The-PageRank-Citation-Ranking-%3A-Bringing-Order-to-Page-Brin/eb82d3035849cd23578096462ba419b53198a556 https://www.semanticscholar.org/paper/The-PageRank-Citation-...
- pcrh 1mo ago
  Inventorship of patents has a clear legal basis, and is quite different from the basis of scientific authorship. The system disincentifies adding "ghost" inventors since incorrect listing of the inventors can invalidate a patent. So, yes, it's likely that the patent attorneys hired by Stanford advised that only Page should be listed as an inventor. Stanford gets its money via the assignment, not the inventor list.
- srean 1mo ago
  If I recall correctly, it was for course project for the class taught by late Rajeev Motwani. Not contradicting your comment, adding more context.

paxys 1mo ago

Thinking of an algorithm in the abstract is one thing, implementing it at scale is another. Yes you could have invented PageRank, but could you also have invented MapReduce, BigFiles/Google File System (GFS), Google Web Server, Bigtable, Protobuf? Then spun up fault-tolerant clusters consisting of cheap commodity PC hardware in an era where AWS wasn't even an idea yet? Then invented the concept of Borg to manage this hardware globally?

blueg3 1mo ago
When Google started, Beowulf clusters were all the rage. So it wouldn't be a big conceptual leap at all. But productionizing it is serious work.
- cmrdporcupine 1mo ago
  There were smart people around thinking about distributed systems to hire in 1996 just as there is now. It's a dream job for nerds. Productionizing it is not at all impossible if you have money to pay people to do it. It's not like there weren't examples of real world distributed job systems out there already (just different). Money you get by being a pair of connected Stanford grads living in the most tech connected place in the world, with Stanford alumni and venture capital connected people all around you. The CS is part is medium-hard. The productionizing part is a hiring problem. Getting the capital is a whole other aspect. I knew lots of smart people in 1996 in the first .com wave. None of us made any money :-) I should have moved to San Francisco in 97 like I was originally planning. Oops. (And no, I could not have done what Jeff Dean & Sanjay did but I would have loved to have tried ;-) )

deleted 1mo ago

[deleted]

ctbeiser 1mo ago

I could not have invented it. I was one year old.

nayuki 1mo ago

Here are two excellent videos that explain and visualize the PageRank algorithm: * [2020-06-17] Spanning Tree - "How Google's PageRank Algorithm Works" (5m16s): https://www.youtube.com/watch?v=meonLcN7LD4 https://www.youtube.com/watch?v=meonLcN7LD4 * [2022-05-23] Reducible - "PageRank: A Trillion Dollar Algorithm" (25m25s): https://www.youtube.com/watch?v=JGQe4kiPnrU https://www.youtube.com/watch?v=JGQe4kiPnrU

saltysalt 1mo ago

Let's not forget that PageRank unintentionally spawned the SEO industry.

targetweb 1mo ago
True
srean 1mo ago
The blame should not rest at Pagerank's feet. The moment the web became a place to make money SEO would have cropped up to target any other ranking system, had that become the defacto standard. In fact the ranking system was not raw Pagerank's, it was one of the many signals used to rank a page. It was the resulting ranking that SEO targets, not Pagerank specifically.
stefs 26d ago
i don't think that's correct. SEO has been popular long before google. many websites were overloaded with long lists of unrelated keywords at the bottom. google became the most popular search engine exactly because those SEO keyword spam lists didn't work anymore and, after a while, they vanished. blogs, on the other hand, worked exceptionally well, because they used to link to other blogs' entries a lot, and blogs also used to be (comparably) high quality content. so the main SEO guideline for getting a good page rank used to be: have a blog/news and publish lots of high quality content that bloggers link to.

notaigenerated 1mo ago

You could also have invented a wheel because it's so obvious.

kqr 1mo ago
The problem with the wheel is not inventing it so much as finding ways to use it. Aside from capstans and pulleys, the wheel is mostly only useful in combination with good infrastructure or complicated machinery.

seebeegeebee 1mo ago

Step 1: be sponsored by the CIA

DANmode 1mo ago
https://qz.com/1145669/googles-true-origin-partly-lies-in-cia-and-nsa-research-grants-for-mass-surveillance https://qz.com/1145669/googles-true-origin-partly-lies-in-ci... More by this author: > a new crowd-funded investigative journalism project, breaks the exclusive story of how the United States intelligence community funded, nurtured and incubated Google as part of a drive to dominate the world through control of information. Seed-funded by the NSA and CIA, Google was merely the first among a plethora of private sector start-ups co-opted by US intelligence to retain ‘information superiority.’ > The origins of this ingenious strategy trace back to a secret Pentagon-sponsored group, that for the last two decades has functioned as a bridge between the US government and elites across the business, industry, finance, corporate, and media sectors. The group has allowed some of the most powerful special interests in corporate America to systematically circumvent democratic accountability and the rule of law to influence government policies, as well as public opinion in the US and around the world. The results have been catastrophic: NSA mass surveillance, a permanent state of global war, and a new initiative to transform the US military into Skynet. https://web.archive.org/web/20190722034225/https://medium.com/insurge-intelligence/how-the-cia-made-google-e836451a959e https://web.archive.org/web/20190722034225/https://medium.co...

joe_the_user 1mo ago

it’ll give you an article on “Hotels for Chickens” if you search “Hotels” because the word matches What if I fricken want fricken "Hotels for Chickens", what about that, huh? I mean, page rank was useful but it was also a step in the direction of "search give you what it think you, not what you asked for" and I think now we can how far and dubious progress in that direction has been.

yungookim 1mo ago

curious, how do you decide on which page to start from? Popular directories?

snarfy 1mo ago

Yes anybody can be in the right place at the right time.

jefflinwood 1mo ago

I actually built this, and shipped it, in 1996, with no knowledge of page rank, citation analysis, or bibliometrics, for an internal/external search engine for the Envirolink web site. Envirolink was a directory of environmental web sites, so they already had a list of URLs to crawl. The reason it was feasible to build was that it was a fairly constrained list of URLs, it wasn't the entire web. I didn't really know what I was doing (I was 17), but it was an awesome unpaid summer internship. There were two parts of the search engine - a crawler and the search engine. Both were written in Perl.

steve1977 1mo ago

No mention of RankDex?

mpalmer 1mo ago

All credit to you for writing this yourself! PageRank, at its core, symbolizes these basic properties. This isn't what "symbolizes" means. You could use "embodies", "satisfies", "models", etc.

yu3zhou4 1mo ago

Also PageRank as a Markov chain [0] https://math.libretexts.org/Bookshelves/Linear_Algebra/Understanding_Linear_Algebra_(Austin)/04%3A_Eigenvalues_and_eigenvectors/4.05%3A_Markov_chains_and_Google%27s_PageRank_algorithm https://math.libretexts.org/Bookshelves/Linear_Algebra/Under...

Animats 1mo ago

As, I think, Page points out in the patent, PageRank's idea comes from Science Citation Index. That was an inverted list of scientific references, where you could look up an scientific paper in an expensive set of bound books and find all the papers in which it was later referenced. You can then use this to see who's getting referenced a lot, which is an ego trip in academia. Academic libraries had copies of that index. Now everybody has that kind of info, but when it had to be done by hand, it was hard. Inverting the huge, sparse matrix of references for PageRank was expensive. Originally, Google did it about once a week. The big breakthrough was when someone (who?) figured out how to do it incrementally at scale.

BiraIgnacio 1mo ago

Related, fascinating foundational work Citation index https://en.wikipedia.org/wiki/Citation_index https://en.wikipedia.org/wiki/Citation_index Shepard's Citations https://en.wikipedia.org/wiki/Shepard's_Citations https://en.wikipedia.org/wiki/Shepard's_Citations

kwanbix 1mo ago

What I always wonder is, how does google knows how to go to abc.com, xyz.com, mypage.com, etc. so it can crawl them?

SJC_Hacker 1mo ago
It starts with seed URLs and fans out from there Back in 1996, this could be done by hand
- kwanbix 1mo ago
  thanks! I imagined it will be something like that. Maybe Google used yahoo index back then as seed?
  - orsenthil 1mo ago
    Yes. I think, there used be a directory of a links published, like an index.

TimCTRL 1mo ago

It's giving "Attention is all you need" but in the year 1996

williamcotton 1mo ago

Here's an in-depth explanation with an interactive demo that I made many moons ago: https://web.archive.org/web/20130728183938/williamcotton.com/pagerank-explained-with-javascript https://web.archive.org/web/20130728183938/williamcotton.com...

kerblang 1mo ago

PageRank is really the beginning of popularity as proxy for authentic/truthful/accurate/etc. Then we move on to upvotes, retweets, likes, listens, views, please-please-please-subscribe-to-my-channel, reviews etc. Thus the continuum of the "social" internet. We tried really hard to make truth into a capitalist endeavor.

1vuio0pswjnm7 1mo ago

Original HN title: "You could have invented PageRank"

1vuio0pswjnm7 1mo ago

Popularity is not equal to relevance Except in the case of advertising Even if, hypothetically, company knows what user is searching for before user finishes typing a query, it does not mean company is going to deliver it to user Company is paid by advertisers, not users Company can deliver something _popular_, call it "relevant", make bank "Popular" grows audience, good for advertisers "Relevant" not necessarily good for advertisers, only good for user Company serves its customers: advertisers Company does not need PageRank to determine popularity, only search traffic (search query data) (Of course a result could be both relevant and popular. But a result could also be relevant and _unpopular_. Company is promotes the former to the exclusion of the later. It profits by selling advertising services to advertisers, not information services to www users)

1vuio0pswjnm7 1mo ago
To stay on top, the company buys search traffic For example, paying vendors to be the "default" search engine