50 ms·

> "Improve your benchmark" only works when you don't have adversaries. If this were true, then there would exist no search engines consistently capable of find

> "Improve your benchmark" only works when you don't have adversaries. If this were true, then there would exist no search engines consistently capable of finding content highly relevant to the user's query -- as opposed to just returning promoted content, which according to your premise is always winning the adversarial battle. Sure, there is an adversarial element, which means there is a constant 'battle' for 'fairness' among results (or whatever your goals are as a search engine). But in reality, search engines as a whole seem to work rather well at finding what we want. Of course results are biased from adversarial pressures (and that's what we're discussing on this thread), but it does seem possible for "benchmarks" to work well in an adversarial environment.

klodolph 9y ago
Search engines work by not just improving the benchmark but also making the benchmark a trade secret. That's what I was saying. And if it's a trade secret I would say that the word "benchmark" is a misnomer. Broadly speaking, a benchmark is a standardized way of measuring something, and if only one company can perform the measurement then it's not really a standard.
deleted 9y ago
[deleted]
drb91 9y ago
> If this were true, then there would exist no search engines consistently capable of finding content highly relevant to the user's query -- as opposed to just returning promoted content, which according to your premise is always winning the adversarial battle. I can’t tell if you agree that google doesn’t return highly relevant content or if you just get better results than I do. Almost all my search results are dominated by low relevance high click through brands, regardless of quality. For example: quora often replaces high quality content with low quality paywalled answers researched from wikpedia or other public resources. 10 years ago you would have gotten metafilter, which is (imho) objectively a higher quality source less hostile to web users.
- electrograv 9y ago
  > I can’t tell if you agree that google doesn’t return highly relevant content or if you just get better results than I do. The degree to which search engines are gamed successfully by adversaries is not the variable I wish to present a strong opinion of here, which may explain your confusion on my position here. Instead, I just wanted to illustrate the logical connection between an adversarial-defensive ability to rank based on [website loading speed], and the ability to rank sites based on [any other factors]. If you can pull off adversarial defense of the latter, you probably can achieve the former with even less effort. That said, I do present to you tentatively the claim that most popular search engines do a decently good job at fulfilling their goals, even if those goals are not in alignment with users' goals in many cases. For example, there are certainly many examples of paid ads given preferential treatment in search results; however I do not believe this constitutes a case of an adversary beating the search engine's metrics. Rather, it is a consensual relationship/contract between the search engine and an advertiser being fulfilled. Whether this contract/relationship is in the best interest of consumers, is another matter altogether, of course.

klodolph 9y ago

Search engines work by not just improving the benchmark but also making the benchmark a trade secret. That's what I was saying. And if it's a trade secret I would say that the word "benchmark" is a misnomer. Broadly speaking, a benchmark is a standardized way of measuring something, and if only one company can perform the measurement then it's not really a standard.

deleted 9y ago

[deleted]

drb91 9y ago

> If this were true, then there would exist no search engines consistently capable of finding content highly relevant to the user's query -- as opposed to just returning promoted content, which according to your premise is always winning the adversarial battle. I can’t tell if you agree that google doesn’t return highly relevant content or if you just get better results than I do. Almost all my search results are dominated by low relevance high click through brands, regardless of quality. For example: quora often replaces high quality content with low quality paywalled answers researched from wikpedia or other public resources. 10 years ago you would have gotten metafilter, which is (imho) objectively a higher quality source less hostile to web users.

electrograv 9y ago
> I can’t tell if you agree that google doesn’t return highly relevant content or if you just get better results than I do. The degree to which search engines are gamed successfully by adversaries is not the variable I wish to present a strong opinion of here, which may explain your confusion on my position here. Instead, I just wanted to illustrate the logical connection between an adversarial-defensive ability to rank based on [website loading speed], and the ability to rank sites based on [any other factors]. If you can pull off adversarial defense of the latter, you probably can achieve the former with even less effort. That said, I do present to you tentatively the claim that most popular search engines do a decently good job at fulfilling their goals, even if those goals are not in alignment with users' goals in many cases. For example, there are certainly many examples of paid ads given preferential treatment in search results; however I do not believe this constitutes a case of an adversary beating the search engine's metrics. Rather, it is a consensual relationship/contract between the search engine and an advertiser being fulfilled. Whether this contract/relationship is in the best interest of consumers, is another matter altogether, of course.