5 ms·

Really crummy title. PageRank is the reason we HAVE a search as powerful as Google, and largely the reason the web is as good as it is today. Raise your hand i

by rco8786 11y ago

Really crummy title. PageRank is the reason we HAVE a search as powerful as Google, and largely the reason the web is as good as it is today. Raise your hand if you want to go back to AltaVista/AskJeeves.

SixSigma 11y ago
To fully remember the evolution of search one needs to have been over 20 in 1995 and on the internet. That's probably quite a small number of people who are now over 20 and on the internet. Do you remember when Yahoo was a website about wrestling ? Or when having a good section in DMOZ was important ?
- recursive 11y ago
  > Do you remember when Yahoo was a website about wrestling ? What? Really? I was around in 1995, and I don't recall that...
  - bryanlarsen 11y ago
    Jerry Wang had a webpage about wrestling that was hosted on the same server as "Jerry's Guide to the World Wide Web", the original name for Yahoo!.
  - kofejnik 11y ago
    i remember it being akebono.stanford.edu
- klenwell 11y ago
  I don't remember Yahoo being a wrestling site. But I was over 20 in '95 and I can remember the internet before Google. It was a bit of a mess. Fun and amazing and kooky. But tantalizingly frustrating if you were trying to do any focused research. I can also remember the moment when I discovered Google. It was reading this article back in 2000 (I could have sworn it was 1998): http://www.newyorker.com/magazine/2000/05/29/search-and-deploy http://www.newyorker.com/magazine/2000/05/29/search-and-depl... If you want to travel back in time to the internet as it was when Google appeared, give it a read. For any gripes I may have about Google's search engine (like the fact I can never seem to easily relocate this article when I want to refer to it), it definitely solved more problems than it created.
takeda 11y ago
I actually miss old AltaVista I always could find what I was looking for with their boolean expressions. Yes, perhaps it was harder to some, but it worked well. With Google, I feel like I have to fight with it. If I'm searching for something obscure or perhaps a word that is misspelled on purpose it thinks it knows better what I'm looking for. It also often returns searches without the word that I searched for and often ignores when I prefix it with + or put in quotes.
- nandhp 11y ago
  Ignoring words in your query that it doesn't like is a relatively new behavior for Google -- in the last three years or so. But yes, there's a reason why I keep a search keyword for Bing.
  - yolesaber 11y ago
    It is one of the most annoying and user unfriendly anti-patterns I've ever come across. It is SO irritating when you are searching for, say, "Hackernews ramen" and you get thousands of results all with the little gray strikethrough text saying "ramen" doesn't appear on the page. If it's not there why bother even returning the result? I know the answer (Ad money / SEO) but it's like one of the basic tenets of information retrieval and they just said fuck it.
    - neolefty 11y ago
      But synonyms! I want to say that if I search for "London Ferris Wheel" and I find "London Eye" without the words "Ferris Wheel" anywhere on the page, that's fine. However, 5 minutes looking through my Google Search History, and I don't see any examples where non-verbatim results would have been useful, so hmm.
      - yolesaber 11y ago
        If Google highlighted that these synonyms appeared in the article, that'd be great. They don't. Thus I, the user, am forced to conclude that the relevancy of the result is pertinent to the other words i.e. the more general search I was trying to avoid by adding in the other word! It's madness I tell you.
cookiecaper 11y ago
I think the problem is that Google puts too much faith in algorithms. A lot of math guys do this; they write an algorithm and are very slow to accept it when the algorithm isn't all-encompassing or totally representative. They need to give more emphasis to human tweaks and consensus, even when their algorithm is trying to figure out consensus, as PageRank does. I'd like to see a hybrid of the high-speed algorithmic scanning of the pages that we see now combined with an army of human reviewers, including super-reviewers who are recognized industry experts, who periodically rate indexed content based on quality. Backlinks and other derived consensus measurements should be given far less weight and a combined algorithmic and human quality rating should be of at least equal importance. So I want to go back to the days of a manually curated web index, combined with the technology needed to make that span out over billions of web pages.
- richardwhiuk 11y ago
  If you think that's a viable proposition feel free to start it. I'm not convinced that the economics add up, nor that recognized industry experts want to spend the majority of their time reading a huge amount of content and rating it for pennies an hour. I don't think Google have too much faith in their algorithm - they know it's flawed. But it's the least worst algorithm anyone has come up with, and adding human tweaks leads it subject to subjective bias.
  - cookiecaper 11y ago
    >If you think that's a viable proposition feel free to start it. Yeah, I don't think it's a bad idea. I don't have the funding to start it, of course, and VCs crap their pants at the thought of anything that has overhead, so it's probably a non-starter. >nor that recognized industry experts want to spend the majority of their time reading a huge amount of content and rating it for pennies an hour. The recognized experts would be paid more than pennies an hour, and they wouldn't need to spend the majority of their time reviewing content. They'd be "super-reviewers", so their opinions would hold a lot of weight. It'd be a way for them to make some extra money without a lot of overhead, something they'd do occasionally for an hour here and there. Honestly the main thing we'd be looking for from these people is information about the cutting edge; things that are trending that we haven't picked up yet, things that are new and thus don't have a lot of consensus markers but are still worth attention, and information about the perception of the content within the industry. That classification can be used to inform on a variety of axes that could be good search parameters. We'd need to make sure we got opposing industry leaders so that the index didn't become solely representative of a single viewpoint. Normal reviewers in a position analogous to a news reporter are more affordable, more consistent, and can classify the majority of content for a sector fine. Maybe Yahoo! could take their niche content mills and reassign the staff to rate pages in their index, then maybe their search will get somewhere. Then you'd have MTurk style reviewers who provide the bulk of the content rankings and just give back a few basic pieces of info. These are the people that would be working for $2-$3/hr or less, at their convenience. All of this is on top of a more traditional automated ranking algorithm that would use consensus markers and computer-perceptible quality markers to rank content. There's not necessarily an obligation that every page is sampled and reviewed by a human. It'd be great if we could get good traffic data too; we'd be able to see where people are actually going instead of just what they put links back to. >But it's the least worst algorithm anyone has come up with, and adding human tweaks leads it subject to subjective bias. The bias is there regardless, it's just filtered through different parameters. This is inescapable. In general, not just in algorithm and computer design, we need less faith in cold systems and more faith in human intervention and judgment. Yes, you have to be aware that any process is subject to gaming, manipulation, or bias, but I think affording sufficient room for human opinion and circumstantial judgment as very highly-weighted inputs prevents most of the egregious failures caused by runaway systems.
sullivandanny 11y ago
The title is about "Google PageRank score," not Google PageRank. PageRank was one part of the reason we have a search as powerful as Google, the use of links to assign an authority score. Another big part was the use of link context -- which isn't part of PageRank but part of the overall search algorithm at Google. PageRank score is when Google revealed those scores to the public for any page. That's not something it had to do, in order to use PageRank as part of it algorithm. But in doing so, it fueled an explosion in link spam.
eevilspock 11y ago
As the author himself (sullivandanny) replied, you misread the title. But I will actually claim that PageRank itself is a problem not just because it is so gamed but also because web page authors link to things they find via Google, creating a positive feedback loop that undermines the very premise of the PageRank algorithm. Search engine research had stagnated because of Google's dominance, and Google itself is not motivated to change, much as Microsoft was unwilling to evolve its cash cow. Rather than innovate in search, it spends most of its resources on ways to shore up its dominance (Google+, Android), fighting the very thing its search engine feeds (Internet garbage), and strengthening its advertising business.
pknerd 11y ago
immune to Google I guess

by rco8786 11y ago

SixSigma 11y ago

To fully remember the evolution of search one needs to have been over 20 in 1995 and on the internet. That's probably quite a small number of people who are now over 20 and on the internet. Do you remember when Yahoo was a website about wrestling ? Or when having a good section in DMOZ was important ?

recursive 11y ago
> Do you remember when Yahoo was a website about wrestling ? What? Really? I was around in 1995, and I don't recall that...
- bryanlarsen 11y ago
  Jerry Wang had a webpage about wrestling that was hosted on the same server as "Jerry's Guide to the World Wide Web", the original name for Yahoo!.
- kofejnik 11y ago
  i remember it being akebono.stanford.edu
klenwell 11y ago
I don't remember Yahoo being a wrestling site. But I was over 20 in '95 and I can remember the internet before Google. It was a bit of a mess. Fun and amazing and kooky. But tantalizingly frustrating if you were trying to do any focused research. I can also remember the moment when I discovered Google. It was reading this article back in 2000 (I could have sworn it was 1998): http://www.newyorker.com/magazine/2000/05/29/search-and-deploy http://www.newyorker.com/magazine/2000/05/29/search-and-depl... If you want to travel back in time to the internet as it was when Google appeared, give it a read. For any gripes I may have about Google's search engine (like the fact I can never seem to easily relocate this article when I want to refer to it), it definitely solved more problems than it created.

takeda 11y ago

I actually miss old AltaVista I always could find what I was looking for with their boolean expressions. Yes, perhaps it was harder to some, but it worked well. With Google, I feel like I have to fight with it. If I'm searching for something obscure or perhaps a word that is misspelled on purpose it thinks it knows better what I'm looking for. It also often returns searches without the word that I searched for and often ignores when I prefix it with + or put in quotes.

nandhp 11y ago
Ignoring words in your query that it doesn't like is a relatively new behavior for Google -- in the last three years or so. But yes, there's a reason why I keep a search keyword for Bing.
- yolesaber 11y ago
  It is one of the most annoying and user unfriendly anti-patterns I've ever come across. It is SO irritating when you are searching for, say, "Hackernews ramen" and you get thousands of results all with the little gray strikethrough text saying "ramen" doesn't appear on the page. If it's not there why bother even returning the result? I know the answer (Ad money / SEO) but it's like one of the basic tenets of information retrieval and they just said fuck it.
  - neolefty 11y ago
    But synonyms! I want to say that if I search for "London Ferris Wheel" and I find "London Eye" without the words "Ferris Wheel" anywhere on the page, that's fine. However, 5 minutes looking through my Google Search History, and I don't see any examples where non-verbatim results would have been useful, so hmm.
    - yolesaber 11y ago
      If Google highlighted that these synonyms appeared in the article, that'd be great. They don't. Thus I, the user, am forced to conclude that the relevancy of the result is pertinent to the other words i.e. the more general search I was trying to avoid by adding in the other word! It's madness I tell you.

cookiecaper 11y ago

I think the problem is that Google puts too much faith in algorithms. A lot of math guys do this; they write an algorithm and are very slow to accept it when the algorithm isn't all-encompassing or totally representative. They need to give more emphasis to human tweaks and consensus, even when their algorithm is trying to figure out consensus, as PageRank does. I'd like to see a hybrid of the high-speed algorithmic scanning of the pages that we see now combined with an army of human reviewers, including super-reviewers who are recognized industry experts, who periodically rate indexed content based on quality. Backlinks and other derived consensus measurements should be given far less weight and a combined algorithmic and human quality rating should be of at least equal importance. So I want to go back to the days of a manually curated web index, combined with the technology needed to make that span out over billions of web pages.

richardwhiuk 11y ago
If you think that's a viable proposition feel free to start it. I'm not convinced that the economics add up, nor that recognized industry experts want to spend the majority of their time reading a huge amount of content and rating it for pennies an hour. I don't think Google have too much faith in their algorithm - they know it's flawed. But it's the least worst algorithm anyone has come up with, and adding human tweaks leads it subject to subjective bias.
- cookiecaper 11y ago
  >If you think that's a viable proposition feel free to start it. Yeah, I don't think it's a bad idea. I don't have the funding to start it, of course, and VCs crap their pants at the thought of anything that has overhead, so it's probably a non-starter. >nor that recognized industry experts want to spend the majority of their time reading a huge amount of content and rating it for pennies an hour. The recognized experts would be paid more than pennies an hour, and they wouldn't need to spend the majority of their time reviewing content. They'd be "super-reviewers", so their opinions would hold a lot of weight. It'd be a way for them to make some extra money without a lot of overhead, something they'd do occasionally for an hour here and there. Honestly the main thing we'd be looking for from these people is information about the cutting edge; things that are trending that we haven't picked up yet, things that are new and thus don't have a lot of consensus markers but are still worth attention, and information about the perception of the content within the industry. That classification can be used to inform on a variety of axes that could be good search parameters. We'd need to make sure we got opposing industry leaders so that the index didn't become solely representative of a single viewpoint. Normal reviewers in a position analogous to a news reporter are more affordable, more consistent, and can classify the majority of content for a sector fine. Maybe Yahoo! could take their niche content mills and reassign the staff to rate pages in their index, then maybe their search will get somewhere. Then you'd have MTurk style reviewers who provide the bulk of the content rankings and just give back a few basic pieces of info. These are the people that would be working for $2-$3/hr or less, at their convenience. All of this is on top of a more traditional automated ranking algorithm that would use consensus markers and computer-perceptible quality markers to rank content. There's not necessarily an obligation that every page is sampled and reviewed by a human. It'd be great if we could get good traffic data too; we'd be able to see where people are actually going instead of just what they put links back to. >But it's the least worst algorithm anyone has come up with, and adding human tweaks leads it subject to subjective bias. The bias is there regardless, it's just filtered through different parameters. This is inescapable. In general, not just in algorithm and computer design, we need less faith in cold systems and more faith in human intervention and judgment. Yes, you have to be aware that any process is subject to gaming, manipulation, or bias, but I think affording sufficient room for human opinion and circumstantial judgment as very highly-weighted inputs prevents most of the egregious failures caused by runaway systems.

sullivandanny 11y ago

The title is about "Google PageRank score," not Google PageRank. PageRank was one part of the reason we have a search as powerful as Google, the use of links to assign an authority score. Another big part was the use of link context -- which isn't part of PageRank but part of the overall search algorithm at Google. PageRank score is when Google revealed those scores to the public for any page. That's not something it had to do, in order to use PageRank as part of it algorithm. But in doing so, it fueled an explosion in link spam.

eevilspock 11y ago

As the author himself (sullivandanny) replied, you misread the title. But I will actually claim that PageRank itself is a problem not just because it is so gamed but also because web page authors link to things they find via Google, creating a positive feedback loop that undermines the very premise of the PageRank algorithm. Search engine research had stagnated because of Google's dominance, and Google itself is not motivated to change, much as Microsoft was unwilling to evolve its cash cow. Rather than innovate in search, it spends most of its resources on ways to shore up its dominance (Google+, Android), fighting the very thing its search engine feeds (Internet garbage), and strengthening its advertising business.

pknerd 11y ago

immune to Google I guess