3 ms·

The problem statement is given the sentiment space of reactions to similar clusters of search queries, minimize negative sentiment across the most clusters give

The problem statement is given the sentiment space of reactions to similar clusters of search queries, minimize negative sentiment across the most clusters given resource constraints (time, cost, CPU). This is a hard problem. There are many search queries, there are many clusters of similar search queries that address partially overlapping (topic,audience segment) pairs. Deriving sentiment scores from these segments is difficult without any feedback mechanisms. Choosing which segments to focus on is difficult or counterintuitive and depends on the topic and audience and characteristics of the audience. Once a topic and audience segment is identified for remediation, the task of effecting change is again topic and audience specific (resistant to automation). This is a hard problem. Neither a brute force "human review squad" approach nor an automated "deep learning" approach will provide 100% coverage through all time. Does that mean that they shouldn't try? No, of course not, and they almost certainly try every day. However, it's inevitable that they will be subject to sniping articles no matter what they do.

andkon 9y ago
I think I agree with you overall — there's no way to perform this task perfectly for all time in such a way that it makes this problem disappear. But I want to push back against the specifics of your argument, namely about the cost and resources involved, as well as the lack of feedback mechanisms. I do that below; I wrote those paragraphs before really giving credit to how hard it would be to comprehensively score audience + topic pairs. That sounds like... exponentially growing complexity. So that approach sounds like a non-starter; maybe an alternative is to look at quality metrics on clusters of search queries as a whole, before diving into seeing if there are problems with individual audiences and those queries. Who knows. There are many search queries to watch, but 'black girls' was persistently problematic for a long time. Noticing when a cluster of queries gets that status might be extremely resource intensive if that status was highly ephemeral; it's not. It's work that can be done asynchronously, and run daily at most. Is this simple? Nothing involving software is, imo. But it's probably much simpler than many other machine learning-driven pieces of Google's product. Also, there are tons of feedback mechanisms available and actively used by Google. Every user interaction with search results is available to Google; a lot of these stand in for quality of result: did the person refine their query after seeing crappy results? Did they click a link and then press 'back' really quick? All of these factors already feature in Google's algorithm.

andkon 9y ago

I think I agree with you overall — there's no way to perform this task perfectly for all time in such a way that it makes this problem disappear. But I want to push back against the specifics of your argument, namely about the cost and resources involved, as well as the lack of feedback mechanisms. I do that below; I wrote those paragraphs before really giving credit to how hard it would be to comprehensively score audience + topic pairs. That sounds like... exponentially growing complexity. So that approach sounds like a non-starter; maybe an alternative is to look at quality metrics on clusters of search queries as a whole, before diving into seeing if there are problems with individual audiences and those queries. Who knows. There are many search queries to watch, but 'black girls' was persistently problematic for a long time. Noticing when a cluster of queries gets that status might be extremely resource intensive if that status was highly ephemeral; it's not. It's work that can be done asynchronously, and run daily at most. Is this simple? Nothing involving software is, imo. But it's probably much simpler than many other machine learning-driven pieces of Google's product. Also, there are tons of feedback mechanisms available and actively used by Google. Every user interaction with search results is available to Google; a lot of these stand in for quality of result: did the person refine their query after seeing crappy results? Did they click a link and then press 'back' really quick? All of these factors already feature in Google's algorithm.