4 ms·

The search quality rater guide says: “Your ratings will not directly affect how a particular webpage, website, or result appears in Google Search, nor will the

The search quality rater guide says: “Your ratings will not directly affect how a particular webpage, website, or result appears in Google Search, nor will they cause specific webpages, websites, or results to move up or down on the search results page. Instead, your ratings will be used to measure how well search engine algorithms are performing for a broad range of searches.” Is this accurate? Or is the guide bending the truth to discourage malicious raters from trying to influence site rankings? Google only uses the rater data to evaluate its search algorithm and doesn’t use the rater data to teach the algorithm? Seems like a missed opportunity. https://static.googleusercontent.com/media/guidelines.raterhub.com/en//searchqualityevaluatorguidelines.pdf#page57 https://static.googleusercontent.com/media/guidelines.raterh...

kappuchino 4y ago
Mostly five raters indenpendently rate the same task and are randomly chosen from a pool. This should reduce SQR conspiracies to near zero. Also, afaik, google engineers use the SQR Feedback to enhance algorithms (example: spam detection) Second: In my active time - long ago - white text was still a thing to make keyword stuffing "invisible". As a result of the SQRs work (and maybe a lot of complaints about bad search results ;-D), the next revision of search ranking "killed" that method - at least for some time - for all pages and not only the rated ones (which were only a sample of all in existance).
lupire 4y ago
Google hires professionals, and surveys the Internet username (Search users and Analytics-integrated website users) to teach the algorithm.
kentonv 4y ago
The quote matches my own understanding of the system from when I worked on Google Search circa 2005. Human ratings were used to facilitate an automated test that could tell you if your new algorithm actually made search results better or not. That let search quality engineers iterate on ideas quickly. But the human ratings were not directly used to produce search results, since that wouldn't scale (only a relatively small number of queries actually had rated results). Obviously 2005 was a long time ago though and everything could have changed since then.

kappuchino 4y ago

Mostly five raters indenpendently rate the same task and are randomly chosen from a pool. This should reduce SQR conspiracies to near zero. Also, afaik, google engineers use the SQR Feedback to enhance algorithms (example: spam detection) Second: In my active time - long ago - white text was still a thing to make keyword stuffing "invisible". As a result of the SQRs work (and maybe a lot of complaints about bad search results ;-D), the next revision of search ranking "killed" that method - at least for some time - for all pages and not only the rated ones (which were only a sample of all in existance).

lupire 4y ago

Google hires professionals, and surveys the Internet username (Search users and Analytics-integrated website users) to teach the algorithm.

kentonv 4y ago

The quote matches my own understanding of the system from when I worked on Google Search circa 2005. Human ratings were used to facilitate an automated test that could tell you if your new algorithm actually made search results better or not. That let search quality engineers iterate on ideas quickly. But the human ratings were not directly used to produce search results, since that wouldn't scale (only a relatively small number of queries actually had rated results). Obviously 2005 was a long time ago though and everything could have changed since then.