4 ms·

PageRank is one factor out of hundreds that determine the rank of a hit in the search results. I'm referring to the algorithm for calculating the rank of a hit

by palish 18y ago

PageRank is one factor out of hundreds that determine the rank of a hit in the search results. I'm referring to the algorithm for calculating the rank of a hit in the search results. If there are hundreds of influences, then it seems overcomplicated.

pierrefar 18y ago
I'm referring to the algorithm for calculating the rank of a hit in the search results. That's not PageRank but the ranking algo. If there are hundreds of influences, then it seems overcomplicated. Well it needs to be a general algo that works with all types of content, all fields of knowledge, work around all known types of spam, in all languages, while working on HTML quality. Complicated? No, necessary.
- palish 18y ago
  That's not PageRank but the ranking algo. That's what I'm referring to! Hehe. I feel insane. Am I really failing at communicating? Let me try again: If the ranking algo considers hundreds of influences, then it is an overcomplicated mess!
  - ntoshev 18y ago
    Actual ranking at the time of your search does not involve too much computation, because a lot has been precomputed and stored in those Google indexes. In theory, you can view every search as a function of your query and all the web documents out there. This function is partially evaluated for a fixed universe of documents and super optimized. Evaluating hundreds of parameters is common sense for machine learning tasks. If you look at the papers explaining how the top Netflix prize competitors achieve their score, this is again what they do: they run literally hundreds of algorithms over the dataset and then blend the results. And this is for a domain where their data is only (user, movie, rating, date) quadriplets.
    - palish 18y ago
      Yeah... But just because a design is also a scientific paper doesn't make it overcomplicated.
      - ntoshev 18y ago
        Don't get your point... Mine was that the solutions to machine learning problems are inherently messy (at least this is the state of the art).
        palish 18y ago
        Typo, sorry. I meant, just because a design is a scientific paper, that doesn't make it a good or simple one.
  - Mystalic 18y ago
    Google keeps it very clean, and they tune the algo every week. But again, it's necessary because relevancy to a human being is continent on THOUSANDS of factors, and they try to incorporate as much of that in as simple of an algorithm as possible.
  - anamax 18y ago
    > If the ranking algo considers hundreds of influences, then it is an overcomplicated mess! "overcomplicated mess" seems to imply "it's a bad solution". While Google's ranking system may be "bad" (they believe that it isn't good enough), "bad" or even "overcomplicated" doesn't follow from "hundreds of influences". Consider the "design" of a common house cat. Its design is the result of thousands of influences. It may be difficult for people to think about systems with hundreds of influences, but that's a people problem and doesn't imply that such systems are inferior to more understandable systems.
    - palish 18y ago
      Housecats weren't designed, they were evolved. That's a big difference. There can be cleanliness and simplicity in design. Also, a housecat's goal is to live and to reproduce, which involves solving many problems, whereas the ranking algorithm's goal is to match text to relevant pages, which requires solving far fewer problems than a living creature, so it's really not a fair metaphor.
      - anamax 18y ago
        > Housecats weren't designed, they were evolved. That's a big difference. What properties of a design do you think are determined by the design process as opposed to its outcome? > whereas the ranking algorithm's goal is to match text to relevant pages, which requires solving far fewer problems than a living creature While "relevance" is the commonly used term, "utility" or "value" would be more accurate. And they're far more complicated than you seem to think. There is no "best" search result for a given query. There's not even a best result for a given person. Add in their context, and you're getting close, but there are still other factors that are relevant to determining "best".
      - bd 18y ago
        I suspect a large part of ranking algorithm's complexity comes also from evolution - they need to coevolve with all SEO stuff. Optimal ranking is a moving target, parasites break your fitness.
Tichy 18y ago
I wouldn't surprised if it is thousands or even millions of influences, many of which might be generated automatically. Part of it might just work like a spam filter, for example, so even single words could be factored into the algorithm. Just guessing, but there could be a rule like "if site contains the string '$$$' reduce rank by 0.5%" and so on - all automatically generated, just as it would happen in a spam filter. Also take a look at SpamAssassin, I have heard it let's you manually combine configure and weigth all sorts of algorithms. At the end of the day, most of Google's employees concerned with improving the search results are probably doing just that most of the time. Even if some Google researcher comes up with a new algo tomorrow, it will probably just be factored in, it won't simply replace the previous algorithms.
deleted 18y ago
[deleted]

by palish 18y ago

pierrefar 18y ago

I'm referring to the algorithm for calculating the rank of a hit in the search results. That's not PageRank but the ranking algo. If there are hundreds of influences, then it seems overcomplicated. Well it needs to be a general algo that works with all types of content, all fields of knowledge, work around all known types of spam, in all languages, while working on HTML quality. Complicated? No, necessary.

palish 18y ago
That's not PageRank but the ranking algo. That's what I'm referring to! Hehe. I feel insane. Am I really failing at communicating? Let me try again: If the ranking algo considers hundreds of influences, then it is an overcomplicated mess!
- ntoshev 18y ago
  Actual ranking at the time of your search does not involve too much computation, because a lot has been precomputed and stored in those Google indexes. In theory, you can view every search as a function of your query and all the web documents out there. This function is partially evaluated for a fixed universe of documents and super optimized. Evaluating hundreds of parameters is common sense for machine learning tasks. If you look at the papers explaining how the top Netflix prize competitors achieve their score, this is again what they do: they run literally hundreds of algorithms over the dataset and then blend the results. And this is for a domain where their data is only (user, movie, rating, date) quadriplets.
  - palish 18y ago
    Yeah... But just because a design is also a scientific paper doesn't make it overcomplicated.
    - ntoshev 18y ago
      Don't get your point... Mine was that the solutions to machine learning problems are inherently messy (at least this is the state of the art).
      - palish 18y ago
        Typo, sorry. I meant, just because a design is a scientific paper, that doesn't make it a good or simple one.
- Mystalic 18y ago
  Google keeps it very clean, and they tune the algo every week. But again, it's necessary because relevancy to a human being is continent on THOUSANDS of factors, and they try to incorporate as much of that in as simple of an algorithm as possible.
- anamax 18y ago
  > If the ranking algo considers hundreds of influences, then it is an overcomplicated mess! "overcomplicated mess" seems to imply "it's a bad solution". While Google's ranking system may be "bad" (they believe that it isn't good enough), "bad" or even "overcomplicated" doesn't follow from "hundreds of influences". Consider the "design" of a common house cat. Its design is the result of thousands of influences. It may be difficult for people to think about systems with hundreds of influences, but that's a people problem and doesn't imply that such systems are inferior to more understandable systems.
  - palish 18y ago
    Housecats weren't designed, they were evolved. That's a big difference. There can be cleanliness and simplicity in design. Also, a housecat's goal is to live and to reproduce, which involves solving many problems, whereas the ranking algorithm's goal is to match text to relevant pages, which requires solving far fewer problems than a living creature, so it's really not a fair metaphor.
    - anamax 18y ago
      > Housecats weren't designed, they were evolved. That's a big difference. What properties of a design do you think are determined by the design process as opposed to its outcome? > whereas the ranking algorithm's goal is to match text to relevant pages, which requires solving far fewer problems than a living creature While "relevance" is the commonly used term, "utility" or "value" would be more accurate. And they're far more complicated than you seem to think. There is no "best" search result for a given query. There's not even a best result for a given person. Add in their context, and you're getting close, but there are still other factors that are relevant to determining "best".
    - bd 18y ago
      I suspect a large part of ranking algorithm's complexity comes also from evolution - they need to coevolve with all SEO stuff. Optimal ranking is a moving target, parasites break your fitness.

Tichy 18y ago

I wouldn't surprised if it is thousands or even millions of influences, many of which might be generated automatically. Part of it might just work like a spam filter, for example, so even single words could be factored into the algorithm. Just guessing, but there could be a rule like "if site contains the string '$$$' reduce rank by 0.5%" and so on - all automatically generated, just as it would happen in a spam filter. Also take a look at SpamAssassin, I have heard it let's you manually combine configure and weigth all sorts of algorithms. At the end of the day, most of Google's employees concerned with improving the search results are probably doing just that most of the time. Even if some Google researcher comes up with a new algo tomorrow, it will probably just be factored in, it won't simply replace the previous algorithms.

deleted 18y ago

[deleted]