15 ms·
Google Has a Striking History of Bias Against Black Girls
- deleted 9y ago[deleted]
- justboxing 9y ago> My first encounter with racism in search was in 2009 when I was talking to a friend who causally mentioned one day, “You should see what happens when you Google ‘black girls.’” I did and was stunned. > These are the details of what a search for “black girls” would yield for many years, despite that the words “p__n,” “p__nography,” or “s*x” were not included in the search box Nonsense. The writer doesn't show the search results for "White Girls" in 2009. I bet it would be very similar. Of course, there's no way to confirm that now, is there, since Google have long fixed their search results to include or exclude NSFW, enable 'safe search' and such, and the author did a search only for 'black girls' in 2009 to "confirm" his hunches.
- fwip 9y agoYou might wanna scroll down, to the section entitled "Beyond black girls."
- curtis 9y agoBack in the 90s, search engines were notorious for serving back lots of porn links interspersed with whatever thing you were looking for. I was unusual (apparently) in that my searches tended to be really specific and I rarely got a lot of porn results. Except one time. I was looking for information on gun violence in Europe and I was getting back not just a few porn results, but lots and lots of them. I was using really simple queries, so I was able to quickly isolate the offending term. It was: "Sweden". (Most likely this was on Alta Vista, not Google, but I don't remember for sure.)
- gaius 9y agoThis is because marketing weasels had just discovered the hottest new tech of the day: meta tags. And immediately set about polluting them. At one point in the mid 90s about half the web that existed at the time was tagged “Anna Kournikova naked”. (Probably many on HN are too young to remember her... imagine Taylor Swift x10, that was her popularity back then)
- deleted 9y ago[deleted]
- tiredwired 9y ago> In 2012, I wrote an article for Bitch Magazine Seems ironic.
- sctb 9y agoAll: we're aware that this article may have an ideological slant that is either positive or negative depending on your own views, but it also has interesting content. Please react to the factual information and steer well clear of any ideological yay-nay combat.
- komali2 9y ago> “p-ssy,” as a noun You are allowed to say fuck on the internet. Also, does it really count if you skip out one letter? I never understood this. You're still all but saying the word. Nobody is being protected here lol. A kid can cycle through the five vowels in about half a second to figure out what the word is, and one of them is also kinda a swear - pissy. EDIT: "n 2012, I wrote an article for Bitch Magazine" now I'm very confused. It's ok if it's a proper noun, I guess?
- grzm 9y agoDifferent publications have different style books. This is Time, which, given it's history as a mainstream magazine, likely has a more conservative editorial style than more recent, online-only publications.
- komali2 9y agoI believe you, I'm just curious why the style guide ever said "swear words are fine as long as you blank out a single letter." Meh, likely the answer is unknown, mostly I'm just commenting
- braderhart 9y agoAt least they have an open source operating system. Let's not forget Microsoft exploiting generational racial poverty.
- SolaceQuantum 9y agoDo we know if Google or tech in general hires black women less than proportional to the US population?
- ucaetano 9y agoBlacks and females are less represented in Google's (and most of tech) workforce, I don't recall seeing specific numbers for black females, but it is trivial to conclude that.
- ceejayoz 9y agoWe do, yes. https://www.usatoday.com/story/tech/news/2016/06/30/google-diversity-numbers-2016/86562004/ https://www.usatoday.com/story/tech/news/2016/06/30/google-d... > Women made up 31% of Google employees in 2015, up one percentage point since 2014, according to statistics released by the Internet giant on Thursday. One in five technical hires were women in 2015, raising the number of women in technical roles to 19% from 18% in 2014 and 17% in 2013. In 2015, women held nearly a quarter of leadership posts at Google, up from 22% in 2014 and 21% in 2013. > Google says it's also hiring more black and Hispanic workers: 4% of hires in 2015 were black and 5% were Hispanic. Hispanic employees in technical roles increased to 3% from 2%. But the increased hiring did not budge the overall percentage of underrepresented minorities in the Google workforce as total hiring rose, with Hispanics making up 3% of the work force and African Americans 2%. Given that women are half the population, and African Americans and Hispanics each make up about 12%, we can state categorically that Google's hiring doesn't match the general US population.
- deleted 9y ago[deleted]
- UncleEntity 9y ago> Given that women are half the population, and African Americans and Hispanics each make up about 12%, we can state categorically that Google's hiring doesn't match the general US population. But if you account for demographics within the industry do the numbers remain biased? For example: men account for less than 10% of nurses so if you were to go to a hospital with 50% male nurses they would be way overrepresented. Statistically speaking of course, I'm in no way trying to give the impression we need to do something about the lack of diversity within the nursing profession. --edit-- ...or maybe we need to do something about the lack of diversity in the nursing profession? Don't actually know how this ball rolls?
- ucaetano 9y agoTry searching Google Images for "brazilian". Her point is quite valid, and also well recognized in at least some part of the tech industry: when you "blindly" optimize for a certain audience, you risk picking up the biases of that audience. So, if you want to appeal to a broader audience than your current one, you might need to manually tune your system to remove some of the biases from your original audience. This is particularly important, if not from a human perspective, at least from a business perspective, when the bias of your current audience is about the larger audience.
- deleted 9y ago[deleted]
- EpicBlackCrayon 9y agoI think the real problem here is that Google, for all of its algorithmic might, is somehow powerless to create algorithms that don't have this problem in the first place. It signals to me that there aren't people thinking about this kind of stuff, both down in the trenches and in the upper echelons. Maybe it's as the author contends, a bigger issue with ethics in tech in general.
- pishpash 9y agoThey're not powerless. They just didn't bother.
- Retric 9y agoThey can't change math because people don't like the results. They can stop relying on algorithms because the underlying data reflects social bias, or use them and reflect society. But, they are always going to upset some group some of the time because social norms are not consistent across society. IMO, attacking Google on this is pointless, deal with the actual problem.
- pishpash 9y agoNever asked to change math or stop relying on algorithms. If you think those are the only solutions you're not thinking hard enough. Google is part of the problem, dealing with the problem involves Google solving its part of the problem.
- komali2 9y agoI like the underlying question that the author brings up - what's google algorithm, and who's responsible for it? In the old days it was "most linked page with those keywords," right? So if you searched "gay man" and got a bunch of gay porn, is it necessarily google's fault that the internet is more gay porn that it is resources for gay men to discuss LGBT issues? Is it google's fault that news agencies more typically report on crime by black people than white (or that black people are more likely to be convinced/arrested in the first place?) I think Google is responsible in 2018, now that they offer extremely tailored results. Duckduckgo, maybe not. But if Google is going to say "we're going to show you listings for restaurants in SF in your simple 'restaurants' search because we know you live in SF," I think they should also say "we're going to show you results for LGBT resources when you search for 'gay men' because nothing about your query indicates a desire for pornographic content."
- deleted 9y ago[deleted]
- andkon 9y agoAlso, how hard is it really to get a sense for the quality of a search and the irrelevance of its results for the vast majority of people? Given how much demographic data Google pulls in from your history, it seems to be not super hard to notice that actual black girls are repulsed by what was shown to them. It also seems easy to post a notice saying: "Hey, searchers like you have found this query to contain many irrelevant or inappropriate results. Beware!"
- amelius 9y agoTaking it a step further, Google could apply this rule: if users really wanted to see (e.g.) porn, then adding that word to the query would show it; so let's keep the results safe by default.
- nickpsecurity 9y agoI like the simplicity of your idea. However, there's articles on porn that are non-pornographic in nature. Usually people discussing or discouraging it rather than using it. To account for that, maybe make it something like "showporn" or a dedicated search page for it. Then, people Googling for information on porn that's not pornographic still wont get hit with it. Whereas, people looking for it using dedicated search page will exclusively see it. This isnt that different from search engines for specialized databases. I used to use meta-search engines that woukd have checkboxes or selection menus to focus search on specific topics or collections. It was very useful. I still do it for technical papers using the "site:" operator to limit search to known-good sites for them.
- krona 9y agoSeveral of the searches given as examples of racial bias are already in a feedback loop; if everyone on the Internet starts writing articles saying 'OMG search for unprofessional hairstyles for work and look racism' then it shouldn't come as a surprise that the first results for otherwise innocuous phrases become skewed, and removing this so-called bias actually requires direct intervention. Although if one actually compares the results for [un]professional hairstyles for work there doesn't seem to be any negative bias; black people are well-represented in both result sets.
- jpm_sd 9y agoUm, it's not "Google", it's "the content of the internet" that's biased - right?
- pishpash 9y agoGoogle contributes to the feedback loop and they are certainly aware of it (the feedback). So "Google is biased" has some validity.
- danso 9y agoBut Google has agency to adapt to the content if needed. If black-hat SEO gets the upper hand and results in an Internet trending toward cheap/fake linkbait, Google (or any competitor) doesn't have to just shrug and give the excuse, "Our shitty search results only reflect the shitty Internet"
- parrellel 9y agoIf Google was still concerning itself with representing the content of the internet, this would be the case, but increasingly Google is tailoring its results for whatever other varied ends. If it is curating the data, then how it curates matters. In this case, they've curated most of the porn from the first page of results, from the looks of it. Not sure how that makes me feel.
- romanovcode 9y agoBasically it's human mind that is biased. Who would've thought?
- politician 9y agoGoogle search results reflect what's in use on the web via PageRank and modern PageRank-like approximations. Google and its human staff does not apply human intellect to the problem of sorting, picking, and ranking each search result. Moreover, Google has included "Safe Search" features for several years that would remove these types of results from search. Before we condemn Google for the collective actions of the broader Internet community of websites, we should take a moment to understand this point. Now, it is possible to create a search engine curated by humans or filters that remove objectionable content, and perhaps that approach might be desirable to some - a "MyGoogle" experience with search results customized based on Google's interpretation (or explicit settings) of the visitor's demographics, political beliefs, trigger words, and purchase history.
- jacquesm 9y ago> Google search results reflect what's in use on the web via PageRank and modern PageRank-like approximations. PageRank is only a fraction of the total inputs to Google search. Once PageRank was described it was subject to being gamed at such a scale that it essentially destroyed the web, ironically making Google that much more a necessity. > Google and its human staff does not apply human intellect to the problem of sorting, picking, and ranking each search result. Yes they do. By tweaking the weights of certain output categories (such as evidenced in the article, a clear drop in the number of pornographic or semi pornographic results as a result of such a tweak) there is a large amount of influence exerted on the results. > Moreover, Google has included "Safe Search" features for several years that would remove these types of results from search. Safe search always was a weird one: You'd expect the opposite, a 'smut search' (Tom Lehrer would have a field day with that one). > Before we condemn Google for the collective actions of the broader Internet community of websites (and the author's step of disabling Safe Search), we should take a moment to understand this point. Before we invalidate the authors point by handwaving and purposefully injecting chaff into the conversation let's try to understand the actual point they are trying to make: That a query for an innocent term such as 'black girls' even with 'safe search off' should not result in a bunch of porn. > Now, it is possible to create a search engine curated by humans or filters that remove objectionable content, and perhaps that approach might be desirable to some - a "MyGoogle" experience with search results customized based on Google's interpretation (or explicit settings) of the author's demographics, political beliefs, trigger words, and purchase history. That's nothing to do with the authors point.
- jacquesm 9y agoSo Google now gives her different results when she makes the same queries. Makes me wonder if this is when she is a logged in user, and what differences there exist between logged in users one to another and non logged in users. When I try it in both logged in and non-logged in mode I get reasonably similar results, the differences are mostly in the ordering. When I do the same on Bing I get about 20% or so pornographic images for both 'black girls' and 'white girls'.
- politician 9y agoI only get wholesome results for "black girls", so maybe Google already puts us in our own echo chambers regardless of the SafeSearch settings.
- jacquesm 9y agoNo, Google has adjusted the weights of that particular output category, see the article linked.
- ksenzee 9y agoThe article points out that Google has already tweaked the result of those particular searches. You're looking at the tweaked version.
- taurath 9y agoIt seems like the main crux of the argument is that being reflective of the culture is also reinforcing it - the fact that say there aren’t as many stock photos of 3 black teens while there is a lot more “mug shot” style photos is probably reflective of the usual portrayal in media, but most people generally agree that the culture shouldn’t be that way. The question is, should search algorithms show what currently is (and therefore reinforce what is), or what the culture aspires to (which in fact more accurately matches the culture in terms of desire and movement)? I think this is a pretty smart example of how the technology we use every day can be passively malevolent - creating hurdles for change rather than enabling better information transfer.
- savanaly 9y ago>It seems like the main crux of the argument is that being reflective of the culture is also reinforcing it I believe expecting Google to change its search results to match an ideological agenda (I'm not using "ideological" in a derogatory sense here) is a case of shooting the messenger. It's the job of humans to make society in the image they desire and the job of the search engine to efficiently and accurately give the search results that people want.
- dnomad 9y agoI don't know if this isn't blindingly obvious by now... but Google already changes its search results to match an ideological agenda. The days of naive algorithms ended with the 2016 election. These days malicious actors will deliberately game any naive algorithm to push their malicious ideological agenda (eg [1]) and the only reasonable thing to do is to push back. (Or I suppose you could let malicious actors overrun your platform.) Google has no choice but to pursue an ideological agenda because it is being targeted by those who have the worst agendas. [1] https://www.theverge.com/2017/10/3/16413082/google-4chan-las-vegas-shooting-top-stories-algorithm-mistake https://www.theverge.com/2017/10/3/16413082/google-4chan-las...
- kolpa 9y agoblack-hat SEO and anti-black-hat-SEO goes back way before 2016
- andkon 9y agoI really appreciate the point: as much as the content of what's on google has nothing to do with google, the prioritization of search results is on them. Whether it's hard or not to avoid perpetuating stereotypes doesn't seem to obviate their responsibility to not perpetuate stereotypes, any more than the fact that Facebook's having so much content flow through their feeds could remove Facebook's responsibility to not help the spread of fake news. Google's search results is really the oldest analog we have for the "machine learning learns racism" quandary, too. This question is just going to get bigger, and its effects much more pernicious.
- daenz 9y agoThis article feels like bait for smart people to ruin their careers with honest responses.
- erikig 9y agoI certainly hope not, there are a lot of great comments above that read as both honest and smart.
- mlazos 9y agoI’m always amazed how often the “surprise” that digital technology is in fact not unbiased and robotic in nature keeps getting written about. Algorithms that optimize for click through rate for ads and search relevancy will inevitably be a reflection of ourselves and all of the shitty things various people believe. >> What we know about Google’s responses to racial stereotyping in its products is that it typically denies responsibility or intent to harm, but then it is able to “tweak” or “fix” these aberrations or “glitches” in its systems. Only Google knows the fate of the code base that has workarounds for all of these edge cases. I think the author is wrong that all of these edge cases could’ve been caught beforehand. Some sure, but I think others would always slip through.
- Hyva 9y ago> I’m always amazed how often the “surprise” that digital technology is in fact not unbiased and robotic in nature keeps getting written about. Algorithms that optimize for click through rate for ads and search relevancy will inevitably be a reflection of ourselves and all of the shitty things various people believe. What you just observed and described is that technology _is_ unbiased and robotic. The bias is in the humans. You can't remove it with a software bandage; the software is working as intended.
- Asdfbla 9y ago> You can't remove it with a software bandage Though some people might argue that the biases software reproduces will in turn reinforce biases held by the users at large. Conversely, you may reduce human bias if you algorithmically reduce the bias in the data (which is possible). Not saying whether Google should, that's a very delicate debate, but it's thinkable. I wouldn't want to be in the place of the person at Google trying to make guidelines for that, but it's definitely doable. For instance, you could probably easily argue that the phrase "three black teenagers" (an example from the article) is pretty neutral in itself and definitely shouldn't be interpreted as a call to produce arrest pictures. In general the term "three [ethnicity] teenagers" could probably be normalized with regards to the setting the pictures show, for instance.
- dmreedy 9y ago
- amrrs 9y agoThere's an interesting concept - Machine Bias discussed in one episode of You are not so smart https://youarenotsosmart.com/2017/11/20/yanss-115-how-we-transferred-our-biases-into-our-machines-and-what-we-can-do-about-it/ https://youarenotsosmart.com/2017/11/20/yanss-115-how-we-tra... Developers who train these models or algorithms should be very well aware that the training data is nothing but a download of a sexist, capitalist, racist - society and that bias if is carried forward - irrespective of anything the future isn't going to be clean!
- caconym_ 9y ago> Images of white Americans are persistently held up in Google’s images and in its results to reinforce the superiority and mainstream acceptability of whiteness as the default “good” to which all others are made invisible. I think that implying intent on Google's part is going way too far. Obviously search results, to a large extent, are going to reflect the society and culture of search engine users. If our society and culture are shitty, should search results pretend otherwise? I'm pretty sure that our society and culture are rife with institutionalized racism and sexism and a whole lot of other badness, and badness of search results seems to me to be a symptom rather than a cause.
- pishpash 9y agoDebiasing signals is a common procedure that signal processors are interested in. Not saying it's an easy or well defined problem, or that the solution will be legal, but dispassionately there is an engineering problem there to be solved.
- crispyporkbites 9y agoThis title is a bit misleading. The article opens with: > My first encounter with racism in search was in 2009 when I was talking to a friend who causally mentioned one day, “You should see what happens when you Google ‘black girls.’” I did and was stunned. but then later on: > Although I focus mainly on the example of black girls to talk about search bias and stereotyping, black girls are not the only girls and women marginalized in search. The results retrieved two years into this study, in 2013, representing Asian girls, Asian Indian girls, Latina girls, white girls, and so forth reveal the ways in which girls’ identities are commercialized, sexualized or made curiosities within the gaze of the search engine. Women and girls do not fare well in Google Search — that is evident. So it's not racism, it's sexism. Or is it? What would have happened if you searched for black boys, white boys etc. in 2009? I bet it wouldn't have been PG friendly...
- themgt 9y agoA Google search for "American inventors": https://i.imgur.com/qw5kfQp.png https://i.imgur.com/qw5kfQp.png A Google search for "American women scientists": https://i.imgur.com/o6mxG56.png https://i.imgur.com/o6mxG56.png Can the media stop doing this thing where they p-hack reality for proof of racist/sexist conspiracies?
- ivraatiems 9y agoHow is that different from what you're doing, cherry-picking two examples that support your point to distract from the overall argument? In fact, the article is much more supportive of its claims, because it's an excerpt from a book that cites peer-reviewed (I believe) research. The point is that there's a trend, not that some search results are weird.
- jacquesm 9y agoYou are missing the point: there is no reason why an innocent query such as 'black girls' should bring up a bunch of porn. And Google seems to agree with this because they have since tweaked things so that that no longer happens.
- deleted 9y ago[deleted]
- ithilglin909 9y agoNo, the point is that the author selectively presents information that bolsters the point she wants to make. I'm pretty certain that if she searched for "<blank> girls" -- white girls, asian girls, etc. -- with safe search turned off, a good number of the results would be NSFW, because the internet is 75% porn (and 25% cat videos).
- jacquesm 9y agoSee the paragraphs titled 'beyond black girls' in TFA.
- 9y ago
- danso 9y agoIt's always bothered me when people, (justifiably) frustrated with the bubble effect that personalization can effect upon search results, want to see Google return to "objective" results. But PageRank/BackRub was never an objective metric, it was always affected by what webmasters chose to link to and the semantic markup they wrote. It seemed to be a good metric for a majority of these things, but assuming few webmasters in the early years were black women or black, then the "objective" metric of PageRank/BackRub is going to be dismal. So yes, I agree that more pressure needs to be put on the algorithm-makers, so that there isn't complacency about accepting the results of "pure" computation. But I feel too much weight is given toward the sentiment of "If Google isn’t responsible for its algorithm, then who is?", as if the problems could be wholly or even substantially fixed through "modifications to its algorithm". The other side of the equation is the data -- and diversity is as important to have in media as it is to tech, even if tech feels like the stronger, more immediate lever in changing things. Now that it's been a long time (in tech time) since 2010/2012, it'd be interesting to hear exactly what modifications/improvements Google made to its algorithm to return such different results for "black girls". Was it a de-emphasizing of negative/outdated sources? Or an arbitrary boost for such entities as "Black Girls Code"? Or was BGC not given an arbitrary boost, but found to have been unjustly ignored/under-weighted when it came to determining SERP? I understand not wanting to reveal details of search engine tweaks the day/month/year after they've been implemented. But half a decade later, hopefully learning more details of these "tweaks" won't leave Google vulnerable to black-hat SEO.
- fmitchell0 9y agoWhat I took from the article were a few points: * Taking a neutral position when building something (its just a tool, it's making decisions based on an aggregate) IS in fact making a decision. State can be -1, 0, or 1 and neutrality yields a certain result. * The local optima of these algorithms are likely a product of local optima to the people and teams who built them. The neutral position seemed like the correct answer. * These articles serve to reinforce the idea that the next frontier is context within algorithmic results. This global optimum can only be achieved through diversity of perspective. Perspectives are a product of one's experience, so the value of diversity can be directly connected to the goal of desiring a optimum solution towards a hard problem. FWIW, I am a black heterosexual engineer and my experiences inform my perspective on these type of problems and blindspots affecting people not like me, especially women. For those wanting to go deeper on the technical, scientific, and mathematical basis for what I'm referring, google 'diversity local optima'.
- dahdum 9y agoMy takeaway was that algorithmic solutions alone will never be enough since they operate at the core on user intent. Males searching for “FOOBAR girls” have, on average, different intent than women. If they represent the majority of searches, algorithms will naturally weigh the results they click more highly. Hyper personalization is seen as an answer, but comes with the downside of a reinforcing bubble and drift to extremes. Human intervention and editing are only a partial solution, bringing up further questions about censorship and which ideology is correct. No easy answers to all of this.
- btilly 9y agoMachine learning systems are good at finding the unconscious biases of large populations. Any change to force notions of political correctness on top of that is very complicated, and makes the algorithm work worse for its intended purpose. Literally, failing to accept the discrimination inherent in the data loses you money directly or indirectly. This is well-known, and not Google's fault. Getting offended at Google for a reality they didn't create and can't control seems silly to me. If you don't like the result, work to change the incentives. Or work to change people's biases. This kind of problem is not new. For example insurance companies have long known that where you live affects how much they are likely to pay out. If they base insurance rates on the data, the result is that your zip code becomes a bigger determinant of insurance rates than your driving record. Which results in very large effective racial discrimination. There are laws limiting that, for example prop 109 in California. However it is an eternal struggle because, in fact, competing insurance companies have a good motivation to have the cost of insurance reflect their best estimate of the expected cost of insuring you, and their cost really is a lot higher if you're in a "black neighborhood".
- interfixus 9y agoWhy are white men stockpiling guns? Why are so many white men so angry? White men must be stopped: The very future of mankind depends [...] The future of life on the planet depends on bringing the 500-year rampage of the white man to a halt. For five centuries his ever more destructive weaponry ... My first three DDG-results on searching 'white men'. I shall now proceed to get over it. Suggest others do likewise.
- social_cooling 9y agoI do not think the word "racism" is a good fit for this. There is no prejudice, or a belief that another race is superior. This is not singling out black women. Any search for "blonde teens" during that time would yield porn-related results too. A search engine just tries to give the most relevant search. Likely, a large part of the population was in search of sex when searching for these type of queries. The "bias" is not with Google, the "bias" is in society, and Google merely reflects this. I can also understand why "three black teenagers" would give mugshots over "three white teenagers". The term "thee black teenagers" just appears more in crime news stories. I think this is a legit societal problem, and changing perception does not change harsh reality. I really am on the fence about the efforts to change this. For instance, the majority of people have a negative view of the "black lives matter" movement, and this is not reflected in the search engines right now. I still think people searching for "black girls" are looking for porn, not "black girls who code"-type stuff, but there is no porn in sight. So you cripple the search engines accuracy in favor of political correctness. I feel a search for "professional" or "CEO" should reflect reality, no more, no less. I don't think a search engine index is the place to start change in society (I do think the number of female CEO's is too low, but this needs fundamental change, not a lick of paint). I also think "cultural appropriation" is antithetical to more inclusion. If white people are restricted from wearing afro hair, dreadlocks, or hoop ear rings, these styles will always remain associated with people of color. The reverse is also true: If no black man dons a business suit, there will be no black men when searching for "professional business man", because there will be no data to support that. I do not think anyone needs protection from search technology. I do not think it affects people's lives to a big degree, if a neural network makes a mistake and labels their partner as a gorilla. These are honest mistakes, due to not having enough training data, that nobody deliberately makes.
- grzm 9y agoIf you don't think representation affects things, please post this under your main account.
- social_cooling 9y agoThat is a good one, but I am not going to. I'll explain why: I have no problem being represented as a privileged white male, part of the dominant culture, set out to suppress others. I do not think this affects my life to a big degree. It may even reflect reality more than the current search for "black girls". I do have a problem staying clear from attacks on my reputation and career, coming from people who feel this is justified when a viewpoint does not agree with them. I've seen people's lives being destroyed for comments similar to mine, people calling up employers or conference hosts in an effort to hurt their income. I think it is just common sense to try to avoid such attacks. No matter my representation, I am not going to post a social security number either (my main account is pseudo-anonymous too).
- yongjik 9y agoThis is a much harder problem that it looks to a casual observer. Just imagine what kind of people would type "black girls" to Google search. Not "black girls outfit", not "black girls social bias", just "black girls". I'll bet at least 50% of the searchers are looking for porn. So even if Google had infinite resources, its choices are: (1) act as a moral authority and deny pornographic contents to those who are looking for them, or (2) show pornographic results to people searching for innocent stuff. ...Or maybe (3) built a perfect profile of every user and just show them what they want. But do we want to go there? (You might be thinking that "black girls" is such an innocent term that Google should be able to determine that it's non-porn, but then the question merely shifts to less popular queries.)
- m_mueller 9y agoIsn‘t this precisely why safe search was invented? I find the „unprofessional hairstyle“ example much harder - you‘d have to first classify all pictures according to minority groups, determine the mix of minority group mix located in the search locale of the user, decide whether the mix should be local, regional, national or international in this case, and then correct the bias in the results accordingly. Seems quite heavy handed and I‘m not sure the results will always please those who seek more social justice. Is Google already doing such a thing now?
- aero142 9y agoYes, but I'm surprised that there isn't some self correction there. My understanding is that one of the primary metrics goole uses is which link you click on last after searching. If the majority of people searching are white and all people want to see people who look like them, then the top result should be white. But I expect that when black women search, they will keep scrolling until they find the picture that they were looking for. Wouldn't Google want to identify each significant subgroup for a query and put each of their respected "best" search result near the top, and not just show the top 10 results for the majority subgroup?
- yongjik 9y ago
- deleted 9y ago[deleted]
- oneplane 9y agoIt's interesting how the article tries to 'blame' an algorithm while it's basically just working fine and simply reflecting the output of culture/country/people. It's not that the software decided that one thing should be the results, and others should not, it just takes what we feed it, and gives it back to us.
- Kequc 9y agoI find the concept fascinating that anyone would think an algorithm is "racist". More or less begging human interference with the algorithm in order to make it less racist, anything anyone does can then be viewed as racist.
- danso 9y ago"Racism" involves prejudice and discrimination predicated on the belief that one race is superior/inferior to another. If a face-detection algorithm fails to easily recognize black or Asian faces, such that people of that ethnicity/race have to go through a more manual/friction-laden process (think manual pat-downs and searches from TSA) to be verified as "human" or "citizen", then how is that algorithm not responsible for discriminatory treatment? I do agree, though, that it is wrong to infer that the algorithms themselves are evil or malicious -- it's possible to be racist without having negative intentions. But what does it say about a society if it continues to optimize algorithms (and related infrastructure) for one group over another?
- oneplane 9y agoOf course those systems 'discriminate', that is what they are designed for. The problems aren't with the system, but with how they were set up, implemented or tested. Say you make a sensor that is supposed to tell the difference between blue, green and purple, but you only test with one shade of blue and maybe two shades of green, you are going to have trouble actually matching your design goals. In the case of the face detection system: they didn't specify and/or test is well enough, which can be due to a number of factors, but will most likely lie with the employees of the company that did the development. If they only have the classic 'pasty white guys' to work with, then it's going to be crap at actually doing face detections for all humans. On one hand you could setup a proper test protocol, on the other hand they shouldn't have taken broad or vague terms when developing/presenting the technology. If you don't have a broad selection of faces to test with, you shouldn't claim you have 'face detection', since you merely have 'detection of faces of the people that work on the project plus anyone who looks like them'. This would be a completely different story if someone writing the face matching code specifically programmed code or wrote configuration data that targets skin tone or geometry of specific groups of people. Some people would like to extend this type of technological issue into the area of HRM and race/gender-bias in society in general, but that is not a technology-only discussion and hardly something the people involved are qualified to argue about. Also: >But what does it say about a society if it continues to optimize algorithms (and related infrastructure) for one group over another? It says that society is imperfect, and that certain levels of xenophobia, bias and true racism exist. Doesn't say much about technology though.
- visarga 9y agoIncidentally I think - this is how you debias an algorithm: you marginalise on all the discrimination criteria and check to see if it is balanced and respectful on each subdomain. Google should have a set of automated tests for that, and it's not that hard - a team of AI and sociology experts could do it. The same type of problem is the capital sin of FB - they amplify bias intentionally, selling us to the highest bidder.
- MichaelGG 9y agoWhat did she expect for "professor style"? If I were hand-delivering results and knew it was from a black, asian, or white person in the US, why would I return different results? Someone entering a general query like that probably wants what the common interpretation is. Which is not going to be a black professor. Searching "professor style for black women" returns the results she presumably thinks should be the default for her. As much as I hate Google, they seem to be delivering exactly what they should. Their job isn't to try to change people's minds, it is to find what they are looking for. What's next? Search for "geek guy" and it's all white. Should we be upset? Someone searching for "geek guy" is probably looking for exactly what Google returns in this case and all the others she highlights. As far as "black girls"... What kind of results are people expecting? Who is actually searching for black girls? Probably people looking for porn... Why should Google return suboptimal results because you got offended? Same for "three black teens". It's not Google's fault that so much news is reporting on crime committed containing that phrase. Did she actually check crime statistics? Maybe "three black teens" get mugshots and commit crimes together at a vastly higher rate than "three white teens". Outside of porn, who is searching for this phrase? If you want "clean" and friendly results, try searching "stock photo three black teens". If anything, this article shows that Google's personalization should detect people that want to search things (that they wouldn't really search for in earnest) in order to be offended then display poor quality results to avoid this "issue". In truth though, I would imagine that if she and other black women often searched "professor style" then didn't click any results and immediately searched for "professor style for black women" then Google would pick up on their results being suboptimal and change. My guess is that no one actually makes these searches outside of making a political point, so personalization cannot fix it.
- Asdfbla 9y agoIt's a difficult discussion because if you look at the literature regarding algorithmic fairness, there really is no measure of equality that satisfies all notions of fairness. If you force representation of seemingly underrepresented patterns in the data then you implicitly engage in social engineering. This might be legitimate and reasonable, but you probably should be transparent about the fact that you "unbiased" your data by externally imposing a certain view of what the "fair" data should look like. Ultimately it's a discussion for social scientists or laywers concerned with discrimination, it's not really in the realm of being fixable by engineering or computer science imho.
- chmln 9y agoGoogle is absolutely at fault here. They are much more involved in tweaking search results than they admit. They are especially slanted on an ideological basis. Try searching "American inventors" - nearly all the people listed are African Americans. I doubt all of this is inadvertent, and its definitely dangerous, given the power of Google. Today they are undoubtedly influenced by "left-wing" ideology, but what about tomorrow?
- danso 9y agoI thought it'd be worthwhile to see how DuckDuckGo handles things since it's ostensibly the search engine without personalization. And it has a different scope and expectations from Google. One thing that comes to mind is how it dealt with the "did the holocaust happen"-type problems by hard coding the instant answers [a], something which Google doesn't admit to doing outright, and wouldn't be scalable anyway. DDG's results for "black girls" is pretty different from Google present-day, though definitely a lot cleaner compared to what the author saw in 2009: https://duckduckgo.com/?q=black+girls&t=hb&ia=web https://duckduckgo.com/?q=black+girls&t=hb&ia=web (sidenote: it's not clear if the author disabled SafeSearch in her 2009 search or not. SafeSearch was definitely part of Google by then, and I can't imagine "sugaryblackpussy.com" getting past that filter. If you turn DDG's Safe Search to "Moderate" -- "Off" returns the same results as "Strict" -- you will get a lot of very NSFW results. I guess this sidenote raises a new set of issues about the author's methodology but will ignore it for the sake of brevity here.) The first result is an article titled "Black Girls Only" in Ebony magazine [0]. Which isn't a bad article (or publication). Maybe people would object that the first result is actually about sexualizing black women (albeit positively). The 2nd result is a lot less promising: "Hot Black Girls (45 pics)" at acidcow.com, which has a higher Alexa ranking [1] than Ebony.com (22K vs 63K), but basically looks to be a clunky imageboard. blackgirlscode.com is #6, followed by blackgirlsrun.com (a running club). The rest of the top results are black girl image sites (photobucket, a Facebook group for Big Beautiful Black Girls). The most notable difference between DDG and Google Results, besides what's #1, is that Google results have a lot more news articles in which "black girls" are in the headline (via NPR, nytimes, and theroot). DDG is a lot more sporadic in comparison DDG for "white girls" is not terribly better in terms of being female-friendly: https://duckduckgo.com/?q=white+girls https://duckduckgo.com/?q=white+girls The first result is to the Amazon listing of a book titled "White Girls", by a well regarded New Yorker critic [2]. But the #2 result goes to urbandictionary.com, and #3 hilariously goes to the Wayans Brothers' classic, "White Chicks". There's a bunch of results relating to "White Girl", the mediocre 2016 film (Wikipedia, rogerebert.com, rottentomatoes.com, etc). And then a bunch of results about white females with men of other races. [a] https://github.com/duckduckgo/zeroclickinfo-goodies/blob/b7a73754210185594d945716aedde5cc8486fcba/share/goodie/historical_events/events.yml https://github.com/duckduckgo/zeroclickinfo-goodies/blob/b7a... [0] http://www.ebony.com/news-views/black-girls-only-503 http://www.ebony.com/news-views/black-girls-only-503 [1] https://www.alexa.com/siteinfo/acidcow.com https://www.alexa.com/siteinfo/acidcow.com [2] https://www.amazon.com/White-Girls-Hilton-Als/dp/194045025X https://www.amazon.com/White-Girls-Hilton-Als/dp/194045025X
- venuur 9y agoApologies for being slightly off topic. While trying to read to article, my browser (Safari on iPhone) was redirected to a scammy looking amazon gift card website. Did anyone else have this problem? I am trying to figure out if it is a problem with Time.com or my phone.
- disgruntledphd2 9y agoLikely your phone, given that no-one else has complained. Sometimes I miss the HN of 2011-12, where literally 80% of the comments were about the design of the site, and how it didn't work on Safari/IE/Firefox/Chrome/Lynx. Not often, though.
- aje403 9y ago"What we need now, more than ever, is public policy that advocates protections from the effects of unregulated and unethical artificial intelligence." Artificial Intelligence has become an alias for "shit I have no knowledge of but would like to pretend to have expertise on that is, at least tangentially, related to a computer"
- SCHiM 9y agoI think the problem is that we're turning the 'learner' around in these situations. It used to be that computer interfaces were static, any shortcuts/quirks it had were learned by the human who then used to their advantage. These days we have systems that is not as smart as us trying to figure us out, with predictable results. Tricks and quirks are ever shifting. A query that used to produce the results yesterday might not function the same two weeks from now. Any skill gained in navigating the system needs to be constantly trained and updated. I believe something similar happened to Hawking, where he had a keyboard that was not smart, but was predictable so that he was the one learning to use the system. He disliked a new version of the software that would not behave in a predictable way because it 'learned'. I can't find the link anymore sadly.
- imh 9y agoThere are some excellent ways of framing this problem already covered in the comments. I'd like to add another. As time goes on, people seem to be treating google searches less as database queries and more as a question/answer system. Google is moving this direction too (e.g. "how tall is the eiffel tower" gives an answer above the web pages). When google was just an excellent database index and relevance ranker, then it was ok for its results to simply reflect the underlying data. If the internet is racist, then looking up things on the internet will find racist things. As time goes on and google purports to give answers and facts and gets more information-ey, they're kinda taking an editorial stance that the results aren't simply relevant webpages, but correct answers in some loose sense. When google tells me the Eiffel tower is 984' tall, and 1063' to the tip, they are endorsing that as The Right Answer. Endorsements come with responsibility. As Google pushes to give correct answers, and people increasingly expect it, it's an endorsement of search results that comes with responsibility to not just reflect the huge corpus that is the internet.
- johnnyOnTheSpot 9y agoThey have a lot fewer crayons in their box than they like to project.
- troupe 9y agoIf you do a search for "three black teenagers" you now see all the top results are screenshots of the mug shots that people saw when they previously searched for the term. So basically the discussion around it makes it more prominent not less. Not saying this is right or wrong, just pointing out the results are basically being reinforced. Now when you do a search for "three white teenagers" you mostly get mug shots of black teens.
- tyingq 9y agoSearching for "Asian girls" or "Mexican girls", etc, produces similar results. I can see the rationale for altering the algorithm for words like "girls". It's also interesting that, for example, the results for "swedish girls" are much tamer. Even though that's sort of a long standing phrase that was very sexual.
- pmarreck 9y agoWith all due compassion, don't "black girls" generally have a hard-enough time with broader acceptance, and this might be what is reflected in google search results? In other words, this is not Google's fault and is mostly just an ugly reflection of our collective problematic values.
- deleted 9y ago[deleted]
- jimrandomh 9y agoI tried replicating the searches described in this article, in an incognito window. Most of the searches described inthe article are quite dated; they're also spread over a significant amount of time. None of the searches I tried came out the way the article described. There was no porn in the results for "black girls"; the third result for "professor style" was a black man, and there was a black woman further down on the first page. Search results for "three white teenagers", "three black teenagers", "professional hairstyles for work" and "unprofessional hairstyles for work" were dominated by links to the very discussion this article is reporting on, which sort of suggests they weren't queries anyone was making to begin with.
- SkyMarshal 9y agoHow do you know the algo/search results weren't tweaked after these critiques went public but before your tests?
- ggggtez 9y agoObviously they were. The point the parent post is making is that some of these results are old (like, 10 years old). Of course the algo would have changed, and the article even says that it did. But if no one is searching it, then of course it's just going to be semi-random results. https://trends.google.com/trends/explore?geo=US&q=%22three%20black%20teenagers%22 https://trends.google.com/trends/explore?geo=US&q=%22three%2... The top related query? "Three white teenagers". It's obvious almost no one searches this term unless they've read the article.
- uncletaco 9y ago> I have continued to search for black girls on a regular basis, at least once a month, and I can report that Google had changed its algorithm to some degree about five months after that article was published. Under the heading "Where are black girls now?" she notes that she doesn't see porn in the results anymore and after I googled black girls I got similar results to what she shows in the picture directly under that, where the first link is black girls code.
- blisterpeanuts 9y ago
- beat 9y agoI want an algorithm to filter out any responses to this that talk about "political correctness".
- ghufran_syed 9y agoI'm not sure how the fact that many web users, many of whom are white, find black girls attractive, is somehow construed to be "bias against black girls"? Unless your job is to see bias everywhere...? Oh.
- fipple 9y agoThe headline should be "Google Search has a striking history of bias against black girls," to clarify between Google the search engine and Google the company that makes the search engine.
- squarefoot 9y agoI'm usually very critical of Google for other reasons (privacy, android etc.) but I don't see them at fault in this case, at least not of being voluntarily racist. Their algorithms "evolution" is based on what people searches and find useful, which means if say someone clicks on a link and stops searching for a while it could mean the user found what he/she was looking for. Now if thousands of people search for "black girls", the search returns say some porn links among others and most users click on them, could we expect the algorithms to ignore porn or rather to slowly adapt to what most users considered the right result? The other points look also valid to me, but again, I don't see why Google should be labeled racist when their engine adapts to what news sources publish. Somewhere in the code there could be an association like 3 teens -> gang and a piece of code skimming news sources that reacts to all articles containing the word "gang"; now just mix the two and what you get? Just try searching for "gang" and look at the colors predominance. Search engines aren't easy to write. If I was an Eiffel language expert I would write a book titled "A tour on the Eiffel language" just to prove that. Try searching that phrase without getting submerged by images of the famous tower. Incidentally, a fun search for "a tour on the Eiffel schnitzel kraut" yielded about 90% food related results and only about 5% tower images although it still contained 100% of the terms for both, but having the word Eiffel not immediately following tour could mislead the search engine, so I changed it to "the tour Eiffel on a schnitzel kraut" and results slightly changed with more towers but also more unrelated stuff like Stonehenge. To me a search engine should return results based on harsh reality if necessary, not church choir dreams; policing search engines results in pursuit of politically correctness would be very dangerous. We won't see that day, nor will our grand-grand-grand-grand-kids, but one day eventually racism will start vanishing from society by itself and search engines will follow, and/or we'll be socially mature enough to ignore anything that could resemble racism to a today's biased mind, giving it even less exposure.
- wrs 9y agoIsn't this basically the same phenomenon (and result) as instant retweeting and the like button? If you simply reflect and amplify the behavior of the general population, you can get some pretty ugly results — which is because the general population has some actual ugly tendencies that are being accurately depicted. So the conversation needs to be about how we should actively bias the algorithm to damp out the ugliness, which is already being done by Twitter, FB, Google, etc. I think we've done the experiment and proven pretty conclusively that a totally unbiased algorithm or information-sharing network is not a good thing for society, unless you want society to be like 4chan. Of course, that immediately opens the can of worms — who gets to decide how to bias the algorithm?
- TangoTrotFox 9y agoSearch for 'white man and white woman.' The reason I mention this is that the article implies this is some sort of result of social bias and stereotyping. As the result of that search shows (it's entirely interracial couples), it most certainly is not. No, it's a result of the fact that Google's search is not particularly smart. It's arguably the best there is, but it still comes down to correlating words to pages and that correlation is far from intelligent. And as an aside, I think the term of bias in algorithms is misused when they are operating as intended. You'd call a poll biased if it implied it was polling group 'x', but did not actually do so due to some inadvertent bias. You wouldn't call it biased if it polled the group it stated it was going to poll and then gave the results - even if those results were not what you personally would want to see. If people are mostly searching for interracial couples when they search for 'white man and white woman' or they are mostly searching for porn when they search for 'black girls' then a biased algorithm would be one that actually returned white couples or non-pornographic results.
- whiddershins 9y agoWithout delving too deeply in to it, it seems google is reflecting existing biases society-wide. This is problematic in the same way as facebook’s echo chamber effect, that it can cause a feedback loop that reinforces and heightens bias and division. Beyond that though, I think it gets gnarly how much we want Google to fix the world by tweaking their algorithms. We obviously want something thoughtful. At the same time I don’t think we should lay all the world’s problems at google’s feet.
- fipple 9y agoI don't think there's a clear answer as to what Google SHOULD show for "unprofessional hairstyles." The article shows a screenshot of it showing a lot of normal-looking black women with natural hair, which of course shouldn't be considered unprofessional. But it's definitely true that society has a racist approach to black women's hair, and doesn't consider it "beautiful" unless it's straightened. Hence nearly all famous black women straighten their hair. Should Google show black women's natural hair in the results for "unprofessional hairstyles" to accurately reflect society's unfair attitudes towards black women's hair? Or should they show actual unprofessional hairstyles like giant mohawks etc.? Or should they show a message saying "No hairstyle is unprofessional. Be who you want to be."?
- deleted 9y ago[deleted]
- lr4444lr 9y agoI'm often reminded of the Oscar Wilde quote: "There is no such thing as a moral or an immoral book. Books are well written, or badly written. That is all." Replace "book" with "algorithm", and you have my sentiments on the issue.
- fipple 9y agoThe world is an unfair place with a striking history of bias against black girls, and Google is built with trillions of data points from this world. Society thinks that black women's hair is ugly and white women's hair is beautiful, and Google, with the power of their machine learning, reflects this unpleasant societal truth back to the user.
- labster 9y agoI think the thesis is wrong here. Google has a striking history of removing bias against black women. People make content for the web. People are racist, and make lots of racist assumptions in their writing. Society is racist. This results in Google's algorithms reflecting the corpus they scanned, and the searches that are made. But the search is just a reflection of us, and how terrible we all are. Google gets embarrassed by the results, and exercises editorial control over search -- which leads to them actively removing racist systemic biases. Sure it's reactive and not proactive, but it's movement in a positive direction. Google likely spends more money on editorial control for issues of racism than for any other thing that doesn't make money.
- shawndrost 9y agoYour second paragraph explains why Google, the search tool, has a striking history of bias against black women, by stating "search is a reflection of [..] how terrible we are". Your third paragraph explains that Google, the company, spends time and money on correcting bias. I don't think that's news to the author, or a refutation. She says tech companies produce tools in environments of racism/sexism, without proper safeguards, and the result is damaging to people of color and women. She's saying Google can and should consider this stuff in advance, and take responsibility for bias in their products (even when it's the reflection of ugliness in third parties as you said). So, I don't see your perspective as being in conflict with her. Except inasmuch as you excuse Google for reflecting our biases. Do you think they should feel OK if their next AI winds up with our biases? If so, why? It doesn't feel like you've explained that.
- brighteyes 9y agoI didn't see that comment as excusing anything. I think we all agree Google should do as much as possible to improve results. But at the same time, if Google gets it right 99 times out of 100, the 100th is what will get an article like this written on it. That doesn't excuse that, it explains why it happens. > Your second paragraph explains why Google, the search tool, has a striking history of bias against black women The article says "Google has a striking history of racism", which you interpret as "Google results have included things which are strikingly racist." That's one interpretation, but it seems equally valid to interpret "Google has a striking history of racism" with "Google has been very racist" (after all, if I said, "Bob has a striking history of racism" you'd be right to assume I think Bob is a racist). I think some of the controversy around this article stems from these interpretations, one of which is a far worse accusation than the other.
- deleted 9y ago[deleted]
- letterdigitdash 9y agoAint nobody got time for them?
- nkkollaw 9y agoThis is ridiculous. This just means that people on the internet who write about black teenagers write about the ones who get arrested. People who create pages about "black girls" create porn content, and most people who search for "black girls" most likely are looking for porn and click on links that talk about porn. It's amazing how people can see racism everywhere.
- viburnum 9y ago"We're only reproducing society's most harmful biases" is not much of defense.
- evo_9 9y agoDuckDuckGo: https://duckduckgo.com/?q=black+girls&t=h_&ia=web https://duckduckgo.com/?q=black+girls&t=h_&ia=web
- bcheung 9y agoSeems like the algorithms just surface already existing patterns in society. They have no inherit bias of their own, they are merely reporting the reality it sees. Is introducing an anti-bias at the search engine level the appropriate solution? It seems a bit too much like Western medicine alleviating the symptoms without addressing the root cause. An anti-bias is just masking it. Also, if the author is a feminist, why she writing for a magazine called "Bitch"?
- sean_anandale 9y agoNow do "white couple", or "american inventors".
- monktastic1 9y agoI see a lot of comments about whether Google results should be reflective of society or unbiased, but little discussion of who gets to define what is unbiased. Since that itself is a judgement call, I don't see how they won't be accused of introducing their own bias.
- question_that 9y agoWhy don't they just pass search results through a second algorithm which removes anything anyone could find offensive? We wouldn't have this problem if registrars did their jobs and protected our feelings by seizing domains.
- AllegedAlec 9y agoWhat a victim complex she has over there...