5 ms·

I used to work at Bing. If you really want Bing to improve, the best thing you can do is just use it: clicks on search results, plus backs and dwell times, are

I used to work at Bing. If you really want Bing to improve, the best thing you can do is just use it: clicks on search results, plus backs and dwell times, are vital training data. Ideally you could use Bing as your default engine, then fall back to Google whenever there's a search that doesn't yield good results. If you have the time, you can also use the Feedback link on the bottom-right of the page to report bad search results; people do actually triage and read those.

seanwilson 7y ago
> clicks on search results, plus backs and dwell times, are vital training data Interesting. Can you explain more on what is tracked after a user clicks a result and how this data is used?
- Analemma_ 7y ago
  (Disclaimer: as I said, I don't work at Bing anymore. Things may have changed in the intervening period and I could be mistaken or plain wrong about things) Obviously ranking search results is immensely complicated with lots of accumulated insider wisdom, but the short answer is that "static" page features (PageRank, how well the query matches text on the page, etc.) are good for pruning down zillions of pages to generate a candidate list of 10 or so results to show the user, but not great at ordering those results, and serving the results in the best order is really important for user satisfaction. For generating the final top-10 ordering, "dynamic" features like how many people clicked that link, how long did they spend on the page, etc. are the most useful. Regarding what is tracked, obviously Bing can only track what you're doing on Bing itself, so clicks on blue links and backs are tracked, but they can't see what you do on the destination page. This data is used to train models/NNs/etc. but the raw click data isn't kept for very long: it's too large and it has to be removed for GDPR compliance.
  - seanwilson 7y ago
    > For generating the final top-10 ordering, "dynamic" features like how many people clicked that link, how long did they spend on the page, etc. are the most useful. Do Google do this as well?
    - Analemma_ 7y ago
      I've never worked there and can't say for sure, but I have to assume that the broad strokes of their search engine are the same as Bing's. Bear in mind my response above was a very very high-level description of a really complex system with tens of thousands of person-years of accumulated knowledge and experience inside it.
nwsm 7y ago
>Ideally you could use Bing as your default engine, then fall back to Google whenever there's a search that doesn't yield good results. This is what I do with DDG. Unless I have a pretty specific search, or want Google's really nice live sports scores widgets, DDG is usually pretty good.
- pheug 7y ago
  DDG doesn't track what you click on, so unfortunately you are not contributing much to improving its and Bing's search quality by using it instead of Bing.
radicalbyte 7y ago
Thanks for posting, it's nice to hear from someone on the inside! Downvoters: why on earth are you downvoting Analemma's post? It's constructive, on point and overall the kind of post which makes HN comments valuable.
- TaylorAlexander 7y ago
  I still don’t understand the hive mind downvote patterns.
- ravenstine 7y ago
  What I'm about to say will probably get downvoted as well, but I've noticed HN getting more polarized in terms of votes in the last maybe 2 years. Because of the way that downvoting works on HN, the few times I use it, my thought is "nobody should see this comment". Because enough downvotes delegitemize and hide a comment, it's effectively telling other people that it's not worth being read. I don't know if that's actually going through people's minds when they downvote, but that's what I think which is exactly why I rarely use it. But too many people now are simply using it on posts they don't like, even if a post deserves to be there. For some reason, it's an unpopular opinion to believe that an idea or opinion you think is wrong deserves to be seen and rebutted rather than silenced.
  - pbhjpbhj 7y ago
    Downvotes for dislike is supported by HN/pg. Their choice of fading out comments, when combined with downvotes-for-opposition is really terrible for a high quality discussion site. Just goes to show the community around a site makes all the difference.
clarry 7y ago
> I used to work at Bing. If you really want Bing to improve, the best thing you can do is just use it: clicks on search results, plus backs and dwell times, are vital training data. A million flies are attracted to shit. I don't believe this sort of training data will ever become useful if it's in the same pool with the rest of the world. See also: voting with your wallet against the tyranny of the majority of uninformed consumers who buy whatever is most marketed. Those pennies don't matter. In fact, I believe this sort of training and optimization for the mainstream plays a role in allowing bad results to proliferate. This is before we even consider the fact that clicking on many results can indicate that they're bad (I click another result because the previous wasn't good), or because they're good (I'm browsing choices). Dwelling long can be bad (crappy & slow site, it takes me long to find the information I want or turn away) or it can be good (I found good stuff and I'm spending a while on it). Whatever conclusion your training system draws might be completely wrong. And probably prone to being gamed.
- mturmon 7y ago
  The above heuristic arguments seem inherently weaker than the direct experience of someone who used to be on the team that improves the results. They are well-funded and should be able to back out the effects you mention.
  - clarry 7y ago
    How are they ever going to know whether they improved results for me or not? You might as well train AI to play a game without ever checking their score. Oh, it's spending 20% more time in each room now and firing fewer bullets than before. Surely it is a better AI now. Sorry, appeal to authority is no argument. Direct experience is valuable if there's an argument or some real scenario we can dissect, otherwise it's nothing more than a baseless claim. Without concrete examples, it's not even an anecdote. And there are plenty of anecdotes about search results for in-depth content becoming harder and harder to find. The person I was responding to posted that they're using this training data to order the top 10 results or so. That's already an indication that it's not very helpful for me. I don't get frustrated if the top few results are in suboptimal order, I get frustrated when I get pages and pages of garbage and irrelevant results and can't seem to get anything useful out of it.
leshenka 7y ago
Yandex does that. At the bottom of search results page there are links to other search engines. So if you aren't satisfied with results, you're one click away from Google. I believe they track these clicks and know which results are bad.

seanwilson 7y ago

> clicks on search results, plus backs and dwell times, are vital training data Interesting. Can you explain more on what is tracked after a user clicks a result and how this data is used?

Analemma_ 7y ago
(Disclaimer: as I said, I don't work at Bing anymore. Things may have changed in the intervening period and I could be mistaken or plain wrong about things) Obviously ranking search results is immensely complicated with lots of accumulated insider wisdom, but the short answer is that "static" page features (PageRank, how well the query matches text on the page, etc.) are good for pruning down zillions of pages to generate a candidate list of 10 or so results to show the user, but not great at ordering those results, and serving the results in the best order is really important for user satisfaction. For generating the final top-10 ordering, "dynamic" features like how many people clicked that link, how long did they spend on the page, etc. are the most useful. Regarding what is tracked, obviously Bing can only track what you're doing on Bing itself, so clicks on blue links and backs are tracked, but they can't see what you do on the destination page. This data is used to train models/NNs/etc. but the raw click data isn't kept for very long: it's too large and it has to be removed for GDPR compliance.
- seanwilson 7y ago
  > For generating the final top-10 ordering, "dynamic" features like how many people clicked that link, how long did they spend on the page, etc. are the most useful. Do Google do this as well?
  - Analemma_ 7y ago
    I've never worked there and can't say for sure, but I have to assume that the broad strokes of their search engine are the same as Bing's. Bear in mind my response above was a very very high-level description of a really complex system with tens of thousands of person-years of accumulated knowledge and experience inside it.

nwsm 7y ago

>Ideally you could use Bing as your default engine, then fall back to Google whenever there's a search that doesn't yield good results. This is what I do with DDG. Unless I have a pretty specific search, or want Google's really nice live sports scores widgets, DDG is usually pretty good.

pheug 7y ago
DDG doesn't track what you click on, so unfortunately you are not contributing much to improving its and Bing's search quality by using it instead of Bing.

radicalbyte 7y ago

Thanks for posting, it's nice to hear from someone on the inside! Downvoters: why on earth are you downvoting Analemma's post? It's constructive, on point and overall the kind of post which makes HN comments valuable.

TaylorAlexander 7y ago
I still don’t understand the hive mind downvote patterns.
ravenstine 7y ago
What I'm about to say will probably get downvoted as well, but I've noticed HN getting more polarized in terms of votes in the last maybe 2 years. Because of the way that downvoting works on HN, the few times I use it, my thought is "nobody should see this comment". Because enough downvotes delegitemize and hide a comment, it's effectively telling other people that it's not worth being read. I don't know if that's actually going through people's minds when they downvote, but that's what I think which is exactly why I rarely use it. But too many people now are simply using it on posts they don't like, even if a post deserves to be there. For some reason, it's an unpopular opinion to believe that an idea or opinion you think is wrong deserves to be seen and rebutted rather than silenced.
- pbhjpbhj 7y ago
  Downvotes for dislike is supported by HN/pg. Their choice of fading out comments, when combined with downvotes-for-opposition is really terrible for a high quality discussion site. Just goes to show the community around a site makes all the difference.

clarry 7y ago

> I used to work at Bing. If you really want Bing to improve, the best thing you can do is just use it: clicks on search results, plus backs and dwell times, are vital training data. A million flies are attracted to shit. I don't believe this sort of training data will ever become useful if it's in the same pool with the rest of the world. See also: voting with your wallet against the tyranny of the majority of uninformed consumers who buy whatever is most marketed. Those pennies don't matter. In fact, I believe this sort of training and optimization for the mainstream plays a role in allowing bad results to proliferate. This is before we even consider the fact that clicking on many results can indicate that they're bad (I click another result because the previous wasn't good), or because they're good (I'm browsing choices). Dwelling long can be bad (crappy & slow site, it takes me long to find the information I want or turn away) or it can be good (I found good stuff and I'm spending a while on it). Whatever conclusion your training system draws might be completely wrong. And probably prone to being gamed.

mturmon 7y ago
The above heuristic arguments seem inherently weaker than the direct experience of someone who used to be on the team that improves the results. They are well-funded and should be able to back out the effects you mention.
- clarry 7y ago
  How are they ever going to know whether they improved results for me or not? You might as well train AI to play a game without ever checking their score. Oh, it's spending 20% more time in each room now and firing fewer bullets than before. Surely it is a better AI now. Sorry, appeal to authority is no argument. Direct experience is valuable if there's an argument or some real scenario we can dissect, otherwise it's nothing more than a baseless claim. Without concrete examples, it's not even an anecdote. And there are plenty of anecdotes about search results for in-depth content becoming harder and harder to find. The person I was responding to posted that they're using this training data to order the top 10 results or so. That's already an indication that it's not very helpful for me. I don't get frustrated if the top few results are in suboptimal order, I get frustrated when I get pages and pages of garbage and irrelevant results and can't seem to get anything useful out of it.

leshenka 7y ago

Yandex does that. At the bottom of search results page there are links to other search engines. So if you aren't satisfied with results, you're one click away from Google. I believe they track these clicks and know which results are bad.