7 ms·

I'm not sure how useful this might be to anyone anymore. Google still uses it for the base ranking. But, the results are then run through a variety of add-on

by tyingq 4y ago

I'm not sure how useful this might be to anyone anymore. Google still uses it for the base ranking. But, the results are then run through a variety of add-on ML pipelines, like Vince (authority/brand power), Panda (inbound link quality), Penguin (content quality), and many others that target other attributes (page layout, ad placement, etc). Then there's also more granular weightings for things like "power within a niche", where a new page might do well for plumbing (because of other existing pages on the site), but wouldn't automatically have any authority for medical topics.

mceoin 4y ago
Is there a way to figure out how a site (or page) is performing against Vince, Panda and Penguin?
ramraj07 4y ago
Where can I learn more about all these models you mention?
- tyingq 4y ago
  I'm not sure. Most of that is from information Matt Cutts used to share when he was the public face of Google's search quality team. Since he left in 2014, they've been very quiet about the space. If you search for things like "matt cutts panda", "matt cutts vince", and so on, you'll see some of what he used to share.
5evOX5hTZ9mYa9E 4y ago
And yet, the results are worse and worse.
- tgv 4y ago
  But does it make them less money?
  - tyingq 4y ago
    "The goals of the advertising business model do not always correspond to providing quality search to users." http://infolab.stanford.edu/~backrub/google.html http://infolab.stanford.edu/~backrub/google.html
    - xtracto 4y ago
      And that's one of the major issues with google search: one metric which would be really good for a search engine is the inverse of the % of advertising related html / javascript in a page. Because that would minimize the utility of SEO for those link farms that only write text to mine ad views/clicks (like food recipes). Alas, google would shoot themselves in the feet by promoting pages that consume less in their own advertising products. If the US government split those two business units into different companies, we could have decent searches (google search could still profit by placing ads on its page results)
- dontbenebby 4y ago
  To be fair, they don't seem to teach the next generation about search operators. I've found just using quotes, OR/AND, and some other basic stuff can often get me what I need. The place where Google does well is that they're a monopoly, so things like image search that require a lot of resources, they are better at, especially paired with their geographic location. (USA! USA! USA!) Edit: I hit enter and the post went out while drafting.
  - heretogetout 4y ago
    Quotes aren't what they once were. They don't guarantee that the term you quoted will be in the results.
    - deleted 4y ago
      [deleted]
    - charcircuit 4y ago
      >They don't guarantee that the term you quoted will be in the results. They do guarantee it. Sometimes Google is buggy and messes up in determining what's page is actually visible on the page compared to just being somewhere in the HTML.
      - not2b 4y ago
        The page Google saw might not be the one you get if you click the link they give you. This kind of thing is in the SEO bag of tricks.
      - heretogetout 4y ago
        That's a loose definition of guarantee, I think. They know what's on the page because they include a brief summary in the results, but even there quoted terms don't always appear. Here's an example someone posted a few months back where google decided the user didn't really want what they said they wanted: https://news.ycombinator.com/item?id=30132344 https://news.ycombinator.com/item?id=30132344
    - 4y ago
- superasn 4y ago
  Yes but it's still the king of the jungle and all the alternates are mostly shit (ddg, bing are far worse). I did learn recently about Kagi on HN and that is marginally better than bing but still doesn't beat Google imo. I just hope there is more and better competition like this for Google soon. That's the only thing that can put google back in its place and put an end to their evil things.
- pelorat 4y ago
  Are you logged in? Google works best when search history is enabled. Just saying.
- soco 4y ago
  The results seem to bring them enough income so I'd say "worse" depends a lot on the point of view.
s1k3s 4y ago
> "power within a niche" In my country we have a website that has complete monopoly over sales of used items, services etc. I'm always amazed that Google is able to determine and put it in top 3 results for a wide majority of searches. "laptop mouse" brings up that site, but "kitchen sink" doesn’t, presumably because people don’t buy used sinks.
- anotherman554 4y ago
  If google returned the used item site for "kitchen sink" and nobody clicked on that result, it would be easy for an algorithm to make the connection that this is a bad result that shouldn't be ranked very high.
TrueDuality 4y ago
I never thought about "brand power" being a metric but that definitely makes sense. I'd be very curious to see what search results would look like if that weight was inverted (more "brand power" the lower the rank). I don't think it would be _better_ but it might make exploring lesser known bits of the internet more accessible.
- tyingq 4y ago
  Google was more cagey about Vince and brand power than they were about other updates. The popular theory was that there was some hand-picked set of "popular brand urls" and that pagerank flowing from those had more weight (beyond the already heavy weight they would have because of their link graph).
ricardo81 4y ago
I've always thought their main intentions would be topical pagerank and author pagerank. Obviously Google have made many leaps with natural language, but measuring 'intent' and inherent trust is still an unsolved one. Google+ may have solved it. I remember hearing about an "author rank" type patent. All the same, the link graph has been truly bastardised because of it being such a prominent facet of ranking for an audience.
creamynebula 4y ago
My search results were a lot better in 2006 when, I assume, they didn't have all these ML pipelines...
- chrisseaton 4y ago
  I don't understand how you're comparing the two. You think the results were better in 2006, but the corpus was also different.
- tyingq 4y ago
  My view of it is that they basically lost the SEO spam wars. Without the ML pipelines, the top results would all be dominated by the smaller, highly skilled, SEO manipulators. They didn't find a way to cleanly excise that spam, so they resorted to a very imperfect hammer...giving a lot of SEO weight to large corporate entities. So basically, a different kind of spam dominates now.
  - jjoonathan 4y ago
    A different, more profitable kind of spam dominates now.
    - Nextgrid 4y ago
      A kind of spam from which Google profits because it often embeds its ads and/or analytics.
  - ricardo81 4y ago
    >smaller Perhaps in other words, market forces (I'm thinking ROI) and economies of scale will win.
    - tyingq 4y ago
      What I was talking about is that there was once a pretty large community of smaller SEO manipulators that did a lot of experimentation on what really worked. Especially in the "black hat" realm. Finding things like areas on big-brand sites that would accept user-generated content, comments, etc, where you could bypass some checks and insert links to sites. They would experiment in a pretty deep way, varying things like the rate of new links, type of new links, variations of anchor text / bare links, and so on. It USED to be very effective. The larger entities don't really have to be that detailed. If you have that brand power, you can just cut partnership/cross-link deals and pay a little attention to things like anchor text in links, contextual text around the link, etc. Edit: Ah, yeah, agreed. What got lost in all this was good content that had no big brand behind it. The indiscriminate hammer Google used to kill off small-guy SEO spam also pushed a lot of actually good stuff, stuff that never did any SEO at all, off the first page.
thayne 4y ago
That doesn't mean it isn't useful, for example for a search engine for an internal wiki where you don't have to worry about SEO or spam. And you said yourself google still uses it, so it would probably still be useful to other search engines, including niche search engines, just not as the only ranking mechanism.
account42 4y ago
Does Google still even use page rank for individual pages? Seem to me that domains are way more important to the base rank.
Throwaway194902 4y ago
I have not used PageRank to rank search results but to detect monitoring sensors in P2P botnets as part of my master thesis and it worked pretty well. It is quite effective in detecting sensors and the only way to prevent detection is to reduce incoming edges, thereby requiring many sensors to monitor a large botnet. So the algorithm is still useful not only for search results.
- tgrzinic 4y ago
  please, can you share a link to your thesis?
  - Throwaway194902 4y ago
    Sorry for the delay. Disclaimer: I turned the thesis in already but I'm still waiting for my supervisor's feedback. So it might be completely wrong and has not been reviewed until now. The thesis itself can be found here: https://git.vbrandl.net/vbrandl/masterthesis/raw/branch/master/report.pdf https://git.vbrandl.net/vbrandl/masterthesis/raw/branch/mast... Some groundwork I built upon: "SensorBuster: On Identifying Sensor Nodes in P2P Botnets" https://git.vbrandl.net/vbrandl/masterthesis/raw/branch/master/references/2017_sensorbuster.pdf https://git.vbrandl.net/vbrandl/masterthesis/raw/branch/mast...
utopcell 4y ago
Whatever you do, whichever ML model you develop, a query-independent ordering of all documents will always be necessary since all distributed IR systems will over-retrieve according to some form of term matching and then apply sophisticated scoring to extract the best documents. You can't score trillions of docs for every query. Google still uses PageRank, but at the risk of stating the obvious, the current PageRank is much more sophisticated than the one found in the bibliography.
- marcosdumay 4y ago
  > You can't score trillions of docs for every query. That completely depends on how you model the queries. It can be a TB sized relation all the way into needing more bits than there are atoms on the universe.
  - utopcell 4y ago
    This is not a generic discussion, we are discussing web search IR in particular, but if you want to be pedantic, the TB sized dataset could be the PageRank values and "scoring" could be the ordering of these values.
    - marcosdumay 4y ago
      The TB dataset is PageRank indexed by search term. You can correlate them more and more and end up with exponentially more data, but even the smallest one (that you'll need a hack of servers to query) is quite useful already.

by tyingq 4y ago

mceoin 4y ago

Is there a way to figure out how a site (or page) is performing against Vince, Panda and Penguin?

ramraj07 4y ago

Where can I learn more about all these models you mention?

tyingq 4y ago
I'm not sure. Most of that is from information Matt Cutts used to share when he was the public face of Google's search quality team. Since he left in 2014, they've been very quiet about the space. If you search for things like "matt cutts panda", "matt cutts vince", and so on, you'll see some of what he used to share.

5evOX5hTZ9mYa9E 4y ago

And yet, the results are worse and worse.

tgv 4y ago
But does it make them less money?
- tyingq 4y ago
  "The goals of the advertising business model do not always correspond to providing quality search to users." http://infolab.stanford.edu/~backrub/google.html http://infolab.stanford.edu/~backrub/google.html
  - xtracto 4y ago
    And that's one of the major issues with google search: one metric which would be really good for a search engine is the inverse of the % of advertising related html / javascript in a page. Because that would minimize the utility of SEO for those link farms that only write text to mine ad views/clicks (like food recipes). Alas, google would shoot themselves in the feet by promoting pages that consume less in their own advertising products. If the US government split those two business units into different companies, we could have decent searches (google search could still profit by placing ads on its page results)
dontbenebby 4y ago
To be fair, they don't seem to teach the next generation about search operators. I've found just using quotes, OR/AND, and some other basic stuff can often get me what I need. The place where Google does well is that they're a monopoly, so things like image search that require a lot of resources, they are better at, especially paired with their geographic location. (USA! USA! USA!) Edit: I hit enter and the post went out while drafting.
- heretogetout 4y ago
  Quotes aren't what they once were. They don't guarantee that the term you quoted will be in the results.
  - deleted 4y ago
    [deleted]
  - charcircuit 4y ago
    >They don't guarantee that the term you quoted will be in the results. They do guarantee it. Sometimes Google is buggy and messes up in determining what's page is actually visible on the page compared to just being somewhere in the HTML.
    - not2b 4y ago
      The page Google saw might not be the one you get if you click the link they give you. This kind of thing is in the SEO bag of tricks.
    - heretogetout 4y ago
      That's a loose definition of guarantee, I think. They know what's on the page because they include a brief summary in the results, but even there quoted terms don't always appear. Here's an example someone posted a few months back where google decided the user didn't really want what they said they wanted: https://news.ycombinator.com/item?id=30132344 https://news.ycombinator.com/item?id=30132344
  - 4y ago
superasn 4y ago
Yes but it's still the king of the jungle and all the alternates are mostly shit (ddg, bing are far worse). I did learn recently about Kagi on HN and that is marginally better than bing but still doesn't beat Google imo. I just hope there is more and better competition like this for Google soon. That's the only thing that can put google back in its place and put an end to their evil things.
pelorat 4y ago
Are you logged in? Google works best when search history is enabled. Just saying.
soco 4y ago
The results seem to bring them enough income so I'd say "worse" depends a lot on the point of view.

s1k3s 4y ago

> "power within a niche" In my country we have a website that has complete monopoly over sales of used items, services etc. I'm always amazed that Google is able to determine and put it in top 3 results for a wide majority of searches. "laptop mouse" brings up that site, but "kitchen sink" doesn’t, presumably because people don’t buy used sinks.

anotherman554 4y ago
If google returned the used item site for "kitchen sink" and nobody clicked on that result, it would be easy for an algorithm to make the connection that this is a bad result that shouldn't be ranked very high.

TrueDuality 4y ago

I never thought about "brand power" being a metric but that definitely makes sense. I'd be very curious to see what search results would look like if that weight was inverted (more "brand power" the lower the rank). I don't think it would be _better_ but it might make exploring lesser known bits of the internet more accessible.

tyingq 4y ago
Google was more cagey about Vince and brand power than they were about other updates. The popular theory was that there was some hand-picked set of "popular brand urls" and that pagerank flowing from those had more weight (beyond the already heavy weight they would have because of their link graph).

ricardo81 4y ago

I've always thought their main intentions would be topical pagerank and author pagerank. Obviously Google have made many leaps with natural language, but measuring 'intent' and inherent trust is still an unsolved one. Google+ may have solved it. I remember hearing about an "author rank" type patent. All the same, the link graph has been truly bastardised because of it being such a prominent facet of ranking for an audience.

creamynebula 4y ago

My search results were a lot better in 2006 when, I assume, they didn't have all these ML pipelines...

chrisseaton 4y ago
I don't understand how you're comparing the two. You think the results were better in 2006, but the corpus was also different.
tyingq 4y ago
My view of it is that they basically lost the SEO spam wars. Without the ML pipelines, the top results would all be dominated by the smaller, highly skilled, SEO manipulators. They didn't find a way to cleanly excise that spam, so they resorted to a very imperfect hammer...giving a lot of SEO weight to large corporate entities. So basically, a different kind of spam dominates now.
- jjoonathan 4y ago
  A different, more profitable kind of spam dominates now.
  - Nextgrid 4y ago
    A kind of spam from which Google profits because it often embeds its ads and/or analytics.
- ricardo81 4y ago
  >smaller Perhaps in other words, market forces (I'm thinking ROI) and economies of scale will win.
  - tyingq 4y ago
    What I was talking about is that there was once a pretty large community of smaller SEO manipulators that did a lot of experimentation on what really worked. Especially in the "black hat" realm. Finding things like areas on big-brand sites that would accept user-generated content, comments, etc, where you could bypass some checks and insert links to sites. They would experiment in a pretty deep way, varying things like the rate of new links, type of new links, variations of anchor text / bare links, and so on. It USED to be very effective. The larger entities don't really have to be that detailed. If you have that brand power, you can just cut partnership/cross-link deals and pay a little attention to things like anchor text in links, contextual text around the link, etc. Edit: Ah, yeah, agreed. What got lost in all this was good content that had no big brand behind it. The indiscriminate hammer Google used to kill off small-guy SEO spam also pushed a lot of actually good stuff, stuff that never did any SEO at all, off the first page.

thayne 4y ago

That doesn't mean it isn't useful, for example for a search engine for an internal wiki where you don't have to worry about SEO or spam. And you said yourself google still uses it, so it would probably still be useful to other search engines, including niche search engines, just not as the only ranking mechanism.

account42 4y ago

Does Google still even use page rank for individual pages? Seem to me that domains are way more important to the base rank.

Throwaway194902 4y ago

I have not used PageRank to rank search results but to detect monitoring sensors in P2P botnets as part of my master thesis and it worked pretty well. It is quite effective in detecting sensors and the only way to prevent detection is to reduce incoming edges, thereby requiring many sensors to monitor a large botnet. So the algorithm is still useful not only for search results.

tgrzinic 4y ago
please, can you share a link to your thesis?
- Throwaway194902 4y ago
  Sorry for the delay. Disclaimer: I turned the thesis in already but I'm still waiting for my supervisor's feedback. So it might be completely wrong and has not been reviewed until now. The thesis itself can be found here: https://git.vbrandl.net/vbrandl/masterthesis/raw/branch/master/report.pdf https://git.vbrandl.net/vbrandl/masterthesis/raw/branch/mast... Some groundwork I built upon: "SensorBuster: On Identifying Sensor Nodes in P2P Botnets" https://git.vbrandl.net/vbrandl/masterthesis/raw/branch/master/references/2017_sensorbuster.pdf https://git.vbrandl.net/vbrandl/masterthesis/raw/branch/mast...

utopcell 4y ago

Whatever you do, whichever ML model you develop, a query-independent ordering of all documents will always be necessary since all distributed IR systems will over-retrieve according to some form of term matching and then apply sophisticated scoring to extract the best documents. You can't score trillions of docs for every query. Google still uses PageRank, but at the risk of stating the obvious, the current PageRank is much more sophisticated than the one found in the bibliography.

marcosdumay 4y ago
> You can't score trillions of docs for every query. That completely depends on how you model the queries. It can be a TB sized relation all the way into needing more bits than there are atoms on the universe.
- utopcell 4y ago
  This is not a generic discussion, we are discussing web search IR in particular, but if you want to be pedantic, the TB sized dataset could be the PageRank values and "scoring" could be the ordering of these values.
  - marcosdumay 4y ago
    The TB dataset is PageRank indexed by search term. You can correlate them more and more and end up with exponentially more data, but even the smallest one (that you'll need a hack of servers to query) is quite useful already.