73 ms·

We read the paper that forced Timnit Gebru out of Google

• sumnuyungi 6y ago
  Is this paper on arxiv? This overview doesn't answer any critical questions. For example, it's easy to fill up 128 references and a reader shouldn't blindly trust a claim that, "The version of the paper we saw does also nod to several research efforts on reducing the size and computational costs of large language models, and on measuring the embedded bias of models." If a key part of Google's claim is that the paper omits relevant research, an author should have simply posted their 128 references and openly asked what work was missing. This whole saga could be easily solved instead of being dragged out for clicks.
  • hezag 6y ago
    No, the paper is not on arxiv > [...] Though Bender asked us not to publish the paper itself because the authors didn’t want such an early draft circulating online, it gives some insight into the questions Gebru and her colleagues were raising about AI that might be causing Google concern.
  • mlyle 6y ago
    > If a key part of Google's claim is that the paper omits relevant research, an author should have simply posted their 128 references and openly asked what work was missing. This whole saga could be easily solved instead of being dragged out for clicks. So, we have on one hand, a researcher who got perilously close to litigation with her employer in the past (IMO because of missteps on both sides). On the other hand, we have an employer that then was skittish about telling her that they didn't want the paper published (to protect the employer's business interests, mostly, it seems, while maintaining a veneer of open research organizations). And resorted to small statements through HR and intermediaries demanding retraction. This relationship has broken down; there's no ready process to tidy up the misunderstandings. I will say that Google's claims to be fostering an open discussion of AI ethics and confronting potentially uncomfortable truths on this path are looking a bit more dubious, though Ms. Gebru doesn't look so particularly easy to work with, either.
• chairmanwow1 6y ago
  It's nice to finally get to see some of the content of the paper and it's awesome that Bender was willing to step up and give some context to world about what the hell it was about.
• nullc 6y ago
  > A version of Google’s language model, BERT, which underpins the company’s search engine, produced 1,438 pounds of CO2 equivalent in Strubell’s estimate So... they're saying it used about $100 worth of electricity. [ https://www.eia.gov/tools/faqs/faq.php?id=74&t=11 https://www.eia.gov/tools/faqs/faq.php?id=74&t=11 ] [ https://www.statista.com/statistics/190680/us-industrial-consumer-price-estimates-for-retail-electricity-since-1970/ https://www.statista.com/statistics/190680/us-industrial-con... ]
  • xnx 6y ago
    (Having not read any of this). Isn't Google committed to be net carbon neutral/negative? If so, does the claimed extra electricity usage matter?
    • nullc 6y ago
      Seems so: https://sustainability.google/commitments/ https://sustainability.google/commitments/
    • xiphias2 6y ago
      The paper should have been probably focused on the title only, because the CO2 usage looks like a publicity stunt. She also didn't subtract the amount of CO2 saved by improved productivity by getting better search results.
      • ardit33 6y ago
        Yep, it feels very disingenuous.... It is like someone complaining that google map uses x, amount of energy, yet the old alternative (printing maps, and getting lost) was much worse CO2 wise.
    • lern_too_spel 6y ago
      Does OpenAI? Does the rest of the industry? Everybody is training larger and larger models, with research groups competing on who will be the first to scale up to some number of parameters. It seems obvious that this would be something an ethicist would be interested in researching to figure out what the impact will be and start discussions about how to offset it.
• sthatipamala 6y ago
  Are these numbers the energy to train a model? The whole point of these new NLP models is transfer learning, meaning you train the big model once and fine-tune it for each use case with a lot less training data. 5 cars worth of carbon emissions is not a lot given that it is a fixed cost. Very few are retraining BERT from scratch. EDIT: The other two points are also disingenuous. * "[AI models] will also fail to capture the language and the norms of countries and peoples that have less access to the internet and thus a smaller linguistic footprint online. " NLP in "low resource" languages is a major area of research, especially because that's where the "next billion users" are for Big Tech. Facebook especially is financially motivated to solve machine translation to/from such languages. https://ai.facebook.com/blog/recent-advances-in-low-resource-machine-translation/ https://ai.facebook.com/blog/recent-advances-in-low-resource... * "Not as much effort goes into working on AI models that might achieve understanding, or that achieve good results with smaller, more carefully curated datasets (and thus also use less energy)." This is also a major area of research. Achieving understanding falls under the purview of AGI, which itself carries ethical and safety concerns. There are certainly research groups working toward this. And reducing parameter sizes of big networks like GPT-3 is the next big race. See https://news.ycombinator.com/item?id=24704952 https://news.ycombinator.com/item?id=24704952
  • marmaduke 6y ago
    > reducing parameter sizes of big networks is the next big race I think this race has been ongoing for a while now
  • rococode 6y ago
    Carbon emissions arguments tend to ignore the value of what's being done as well. BERT and other transformers were meaningful experiments that were valuable in furthering a major research direction and enabling more effective consumer and business applications. In that sense, it's like any other company doing R&D - of course energy will be used and of course there will be some inefficiencies. I think it's quite misleading to compare the energy usage of an industry-wide research effort to individual consumption. The graphs look bad - "wow, 626,000 lbs! that's 284 metric tons of CO2! a plane flight is way less!" - but there's a fundamental difference between "progress on a problem being worked on by thousands of highly-paid researchers" and "I bought a car". Meanwhile, the worst power plants are generating on the order of 10+ million tons of CO2 every year. There are at least a dozen of these in the US alone. Car factories are emitting hundreds of thousands of tons of CO2 (Tesla is somewhere around 150,000 tons a year, apparently, and it's designed to be efficient). Perhaps activism around CO2 emissions in ML training might be better focused on improving the efficiency of those things instead, seeing as a 1% improvement would outweigh the entirety of the NLP model training industry. It's certainly good to keep in mind the energy costs of training in case things balloon out of control, but right now the costs relative to the results seem small and not worth highlighting as some forgotten sin.
• deleted 6y ago
  [deleted]
• treis 6y ago
  This doesn't strike me as anything that needs to be buried. The energy argument is tenuous and at this point models use relatively little compute power. The language bias is a better tack but I don't think this is earth shattering to anyone. Most of the internet is from Western sources and some fraction of that is racist/prejudiced. I think this is obvious to anyone who has ever used the internet.
  • creato 6y ago
    It sounds like it should be buried because it's clickbait headline generating fluff that only would get attention because of where the authors come from. And I say that after reading a summary that sounds like the authors had a positive opinion of the paper. Google (probably) wanted to bury it because many of those clickbait headlines would have been negative to google.
    • angry_octet 6y ago
      You should read some other ethical AI research. This is par for the course. We're still suffering talks about killer robots.
      • wan23 6y ago
        We already have drones with cameras, drones with guns, and drones that are autonomous. How long until someone combines these?
        • bigbubba 6y ago
          A few decades ago? What is a fire-and-forget guided missile but an autonomous suicide drone? Some missiles, particularly anti-ship missiles, even use computer vision in the air to identify objects and pick targets. Mark 60 CAPTOR naval mines sit underwater listening to nearby ships. When a noise is classified as an enemy ship, the mine releases a guided torpedo to kill it. Of course autonomy in naval mines is nothing new; the most basic sort 'decide' to kill any object that bumps them too hard. The innovation is making them discriminate between enemy ships and everybody else.
• godelski 6y ago
  While this is a lot of electricity and CO2 to consider, is it fair to compare them to cars? I don't think this so much means that one should get rid of training these big models but rather do their best to ensure that the data centers that these models are trained on are getting their electricity from mostly 0 emission sources. This actually seems like a doable task since you don't need to be physically near your compute cluster. I think Google and Facebook could also turn this into a PR move by putting some research money into green tech (ambiguous).
• juanbyrge 6y ago
  I was a little disappointed in the quality of this research. I can see where Google is coming from. Some of these topics seem regurgitated (e.g. bias in online text) and some are just irrelevant (CO2 emissions). Training language models does not contribute in any meaningful way to global CO2 emissions. Overall not a very strong paper in contrast to the other reactions I've been seeing online.
  • hezag 6y ago
    > I was little disappointed in the quality of this research But the research is not even published yet
    • fastball 6y ago
      Kinda the point though isn't it? We're only reading a summary of the paper because apparently the authors aren't confident enough in its quality to release it publicly. You can't claim that Google dismissed this paper out-of-hand while simultaneously saying "oh, but it's too much of a draft to release publicly". Uh, if it was too much of a draft for the public why shouldn't it be too drafty for Google? Are we really pretending that Google has lower standards of quality than the general public?
      • viktorcode 6y ago
        That, or Google is censoring the paper because of the conclusion it comes to, which doesn't align with their business interests.
        • fastball 6y ago
          I’m taking the article at face value, which states the authors think it’s too much of a draft.
        • juanbyrge 6y ago
          If a company is paying a researcher a lofty salary (I am guessing ballpark mid six figures), and provides that researcher virtually unlimited tools and resources, shouldn't that company have a say in what material gets published by the researcher? Why shouldn't it have the ability to restrict research of dubious quality that makes borderline incorrect conclusions about the company?
• dang 6y ago
  The threads so far on this ongoing story: https://news.ycombinator.com/item?id=25307167 https://news.ycombinator.com/item?id=25307167 https://news.ycombinator.com/item?id=25292386 https://news.ycombinator.com/item?id=25292386 https://news.ycombinator.com/item?id=25285502 https://news.ycombinator.com/item?id=25285502 https://news.ycombinator.com/item?id=25289445 https://news.ycombinator.com/item?id=25289445
  • adrian_mrd 6y ago
    Hi dang, I’ve recently noticed that this commonly occurs ie multiple relevant threads that make up the top 50 of HN over say a 72 hour period, all related or rifts on a similar discussion. Wondering if there is a way to group them as part of the same topic/submission? (thus saving you the manual work of posts like this). I appreciate this would (i) require a code change and (ii) would shift the HN model from individual threads based on 1 URL submission, but just thought I’d suggest it to the HN brains trust. In other news: Keep up the good work that you do on HN to give this online community ‘structure’.
    • adrian_mrd 6y ago
      One potential implementation: at the top of each thread, after the metadata for the thread itself and before the comment box, are a list of ‘Related discussions’ with a bulletpoint list of related HN threads.
      • tsjq 6y ago
        That will be really good
      • CydeWeys 6y ago
        Unfortunately, PG is of the opinion that he "finished" HN over a decade ago, which is why we don't get any iteration on the feature set here, no matter how useful it might be. I agree that this would be a good addition but I don't see it happening with this mindset. https://news.ycombinator.com/item?id=18183822 https://news.ycombinator.com/item?id=18183822
• chaganated 6y ago
  > There have been a few high-profile cases, such as the college student who churned out AI-generated self-help and productivity advice on a blog, which went viral. This is how much of the internet has felt for a long time. After this nugget, I now wonder if Vox is just a model trained on Piketty and Tumblr. edit: Also not sold on the CO2 argument. Too many variables! Nerds will calculate and re-calculate such things, with the result jumping all over the place, swearing that they've gotten it right--this time! No humility, in spite of the odds.
  • mhh__ 6y ago
    Vox are economically usually a lot more neoliberal than Piketty.
• mimikatz 6y ago
  Is this what esteemed PhD AI ethicists spend their time on? It sounds like a useless grad school paper. I think much less of everyone now.
  • emerged 6y ago
    The mistake was expecting much from “PhD AI ethicist” - a title Orwell accidentally left out.
    • HarryHirsch 6y ago
      Medical ethics is a serious subject, and then there was Asilomar on the ethics of genetic engineering. Just because this AI ethicist isn't one, it doesn't follow the field is worthless.
      • emerged 6y ago
        I’m going to stand by my post even though AI ethics should be a genuine field. Call me cynical.
        • HarryHirsch 6y ago
          The Asilomar Conference on Recombinant DNA was a high-caliber event. The paper here, done at Google, a leading AI company isn't, and the ethics-washing on view here is deeply concerning.
          • emerged 6y ago
            That’s the crux of the problem. Top tier academia and ethics both attract the absolute best and the absolute worst. So I suppose we need both the critical cynicism and the optimists in threads like these.
  • LudwigNagasena 6y ago
    This is exactly what I would expect from an “AI ethicist” to be honest.
• machinelabo 6y ago
  I have a genuine question: If a person of color (I am myself non-white) is not performing, what does it take to fire them without the entire world playing the race card on you? Are we creating a society that makes it impossible to fire a person of color? You know there are bad apples in every race, right? How do we handle such scenarios? Seems unfair to me, myself being a person of color - I don't want the world to treat me like some kind of a hero for being non-white / minority. I want fairness and it is frankly offensive. I am not pleased with the way we're treating each other. It's supposed to be equal opportunity. I also want us to have scientific discussion about gender differences (backed by research) and other difficult conversations. Nature doesn't give a fuck about any of this - if our goal is to uncover the way mother nature works, we're going to have to meet difficult truths and not be afraid of it.
  • csnover 6y ago
    You’re going to be downvoted, not because of your questions, but because you decided to put “please bring the downvotes” at the end of your comment. Why would you undermine yourself by sticking such an antagonistic comment at the end of your post?
    • machinelabo 6y ago
      I removed it. I know its going to be downvoted though. Because it is going against the grain on HN. But, I am being respectful. We've created an atmosphere of fear. I don't feel comfortable voicing my opinions even after being anonymous on the internet. That's pretty fucked up.
      • aaomidi 6y ago
        Considering a Pennsylvania law maker made fake Twitter accounts to write "as a gay person", when you start saying you're a non PoC and conflating what happened with Timnit to "firing badly performing PoC"s you do not at all sound genuine. There is no way to prove you are actually PoC. And even if you did, it's incredibly unrelated to the matter at hand. No one has even claimed she was fired for bad performance and dozens of Google employees have said in no part of their process was academic rigor taken into account. What you're doing is the literal definition of concern trolling.
• rayiner 6y ago
  > An AI model taught to view racist language as normal is obviously bad. The researchers, though, point out a couple of more subtle problems. One is that shifts in language play an important role in social change; the MeToo and Black Lives Matter movements, for example, have tried to establish a new anti-sexist and anti-racist vocabulary. An AI model trained on vast swaths of the internet won’t be attuned to the nuances of this vocabulary and won’t produce or interpret language in line with these new cultural norms. This is a pretty superficial take on what is an extremely interesting sociological topic. (To be clear, I’m referring to the article, not the underlying paper which we don’t have.) Obviously just because social movements “have tried to establish ... vocabulary” doesn’t meant that vocabulary has become a “new cultural norm.” Plenty of such efforts end up being cultural dead-ends. Take for example a term like “LatinX.” This term has been proposed and is used by certain people, but is extremely unfamiliar and often alienating to Latinos themselves: https://www.vox.com/2020/11/5/21548677/trump-hispanic-vote-latinx https://www.vox.com/2020/11/5/21548677/trump-hispanic-vote-l... (“[O]nly 3 percent of US Hispanics actually use it themselves.... The message of the term, however, is that the entire grammatical system of the Spanish language is problematic, which in any other context progressives would recognize as an alienating and insensitive message.”). The article hand-waves away a deeply interesting question: What should an AI do here? Should AI reflect society, or be a vehicle for accelerating change? It seems at least reasonable to say that the AI should reflect what people actually say, in which case a big training dataset is appropriate, instead of what some experts decide that people should say. In some contexts, for example with “LatinX,” researchers seeking to enhance inclusivity could instead end up imposing a kind of racist elitism. (People without college educations—which disproportionately comprises immigrants and people of color—tend to be less knowledgeable about and slower to adopt these changes in vocabulary.) The paper seems to imply that AIs should not reflect “social norms” but that training data should be selected to accentuate “attempt[ed]” shifts in such norms. Maybe that’s true, but it doesn’t seem obviously true. To return to the example above, is some Google AI generating the phrase “LatinX” (which 3/4 of Latinos have never even heard of: https://www.pewresearch.org/hispanic/2020/08/11/about-one-in-four-u-s-hispanics-have-heard-of-latinx-but-just-3-use-it https://www.pewresearch.org/hispanic/2020/08/11/about-one-in...) in preference to “Latino” or “Hispanic” actually the desired result?
  • 6y ago
• TnkBldr 6y ago
  what’s the energy expense of training GPT-3? It should be very high. Eager to know the upside of such models. Environmental and social impacts of tech are becoming more direct and mainstream in today’s world!
  • Baetylraig7 6y ago
    How exactly is the environmental impact of tech becoming more "direct and mainstream" [citation needed]? Data centers around the world account only for a tiny fraction of electricity consumption or carbon emissions. Do you even account for the reduced car and air travel due to remote working and online shopping? How are advanced few-shot learners like GPT-3 even remotely a problem, training less models is somehow worse? Do they even know what they are taking about? Its all very confusing, but a lot of the work done by grievance studies becomes immediately easier to understand once you realize they are arguing in bad faith.
• TnkBldr 6y ago
  What is the energy expended on training GPT-3? Eager to know the upsides of such models. The environmental and social impacts of tech are more direct and mainstream in today’s world.
  • baskire 6y ago
    You realize the energy and co2 production to release the paper is greater than the training itself.
• quantumwoke 6y ago
  Dr Gebru's research AFAICT shines a valuable light on some interesting ethical problems. Yet the response so far from Google and even HN has been one of censorship and suppression of Dr Gebru's free speech. How can we reconcile free speech and Google's right of review to her research?
  • lopmotr 6y ago
    Not sure the details of her employment arrangement, but usually work you do for your company belongs to the company and you don't have any right to freely publish it. Seems she was able to publish it anyway, so that's pretty good, isn't it? She's got more than the usual helping of free speech. Where's the censorship or suppression?
    • DenverCode 6y ago
      It was published? Where?
• hezag 6y ago
  I did not read the paper (just like most people here), but by the title — “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” — it does not look like the CO2 emissions thing is the main topic of this research. BTW, "Stochastic Parrots" is a very descriptive name for the problem > Moreover, because the training datasets are so large, it’s hard to audit them to check for these embedded biases. “A methodology that relies on datasets too large to document is therefore inherently risky,” the researchers conclude. “While documentation allows for potential accountability, [...] undocumented training data perpetuates harm without recourse.” Since these models are being applied in a lot of fields that directly affects the life of millions of people, this is a very important and underdiscussed problem. I really want to read the paper.
  • sanxiyn 6y ago
    > Since these models are being applied in a lot of fields that directly affects the life of millions of people In particular, it is being applied right now to rank Google search results, and probably responsible for lots and lots of Google's profit. You should be skeptical of Google's appraisal of the paper that is material to Google's profit.
    • visarga 6y ago
      I bet you don't need to evaluate the NLP model when you have direct access to the end result, Google Search.
  • visarga 6y ago
    > BTW, "Stochastic Parrots" is a very descriptive name for the problem Meh, both parrots and language models are inferior to humans in producing language, but one is more useful than the other. And real parrots are also stochastic, like all living things.
    • basicneo 6y ago
      Analogies aren't meant to withhold careful scrutiny. They're suggestive.
• ChrisArchitect 6y ago
  What was this article from technology review - yes it gave insight into the contents of the paper but then barely made any conclusion or added anything about what that meant to the situation other than the very end saying the obvious maybe this cuts into Google's 'cash cow'? Bah
• fastball 6y ago
  If the authors aren't confident enough in their paper to release it publicly, why would I want to read someone else's (presumably inferior) summary of it?
  • amf12 6y ago
    This is a good point that I did not think of. Google maintains the paper was not up to their standard. The authors submitted the paper to a conference. And now, one of the author asks MIT Tech Review to not publish it because it is an early draft. Gives some credibility to Google's claim IMO.
• actuator 6y ago
  > It will also fail to capture the language and the norms of countries and peoples that have less access to the internet and thus a smaller linguistic footprint online. The result is that AI-generated language will be homogenized, reflecting the practices of the richest countries and communities. It seems amusing to read this when the same person and her followers hounded Yann LeCun for pointing out the same thing with image models. Anyway, this seems interesting. But I am not sure how you solve this. Do we take representative dataset according to population of a place? Also, assuming this is limited to a single language. How can AI generated language account for nuances from regional differences at the same time in a common model. Isn't what the author asking for here is kind of train, en_US, en_GB, en_IN separately here. For things like completion don't language models already account for this?
  • lopmotr 6y ago
    That's a problem for lots of technologies. The native language in my country hardly has any literature published in it because it's so obscure. How should the printing press solve this unfairness? The internet in general also doesn't use it much at all, neither do all the popular movies and TV series. They don't even get subtitled. Apple Maps can't pronounce the street names correctly for navigation either. It's pretty hopeless but does it really matter? Maybe if you have an unpopular language, that's just unfortunate and please encourage your kids to learn English (which they probably do at school just about everywhere anyway) so you don't perpetuate the same problem to the next generation.
• karmasimida 6y ago
  Honestly question to fellow HN commenters: what is the popular take on energy efficiency against entertainment industry? Where do people draw the line, regarding what and what not should be an ethnical use case for energy?
• KingOfCoders 6y ago
  See the other thread were one mentions that with reCaptcha Google learns that all cabs are yellow (and traffic lights hang over streets - they don't here). But I guess everyone only sees the blind spots of other people.
  • angry_octet 6y ago
    You're assuming that reCaptcha is training some Google algorithm. But in fact it has trained you -- to recognise US style dangly traffic lights, yellow school busses and cabs, pedestrian cross walks -- and this makes me wonder if reCaptcha does use the same image set for all different locations. Are people in Lagos, Pune and Tashkent routinely failing reCaptcha at higher rates because they don't watch US television? How fast do they learn what a school bus looks like?
    • KingOfCoders 6y ago
      I knew those already from being flooded for 40+ years with Hollywood films (all of them) and sitcoms, I know Cosby, Malcolm in the Middle, Beverly Hills 90210, Miami Vice, the Fall Guy, Seinfeld, Friends, Lou Grant, Mary Tyler Moore Show, Diff'rent Strokes, Cheers, Golden Girls, MAS*H (the best), Married... with Children, Three’s Company, King of Queens, Family Matters and on and on and on. Basically I know the US better - or the image it projects - than my own country. Funnily it was a huge letdown when I visited for the first time decades ago, as it felt just like a sitcom and there was nothing new really.
      • TotallyHaram 6y ago
        Those images being exported to the planet are not representative of the USA. Hollywood is mostly Jewish and Homosexual, those are the products of these minds instead. I doubt actual American programming would resemble any of these sitcoms.
    • Macha 6y ago
      Even in Europe I sometimes find reCAPTCHA more difficult for being "too American". Maybe we're categorised as "close enough" and there are south American or African datasets but I suspect not.
• _cs2017_ 6y ago
  Despite what both sides claim, IMHO papers like this are primarily PR, ideology, and politics, rather than science and technology. This is akin to a speech writer working for a politician, writing a piece that disagrees with the party platform, and refusing to fix it when asked.
  • visarga 6y ago
    yes, basically it's activism disguised as research
  • kuschku 6y ago
    Well, she did Research while Google expected a PR piece. And it's obvious she couldn't fulfill Google's expectations, as the whole point of an ethics committee is to be critical, not just doing rubber-stamping.
  • DubiousPusher 6y ago
    Can I postulate that the problem with these kind of positions goes even deeper? Let's say we had a wonderful company which would create an unimpeachable ethics board and which enthusiastically endorsed the findings of said board. This is good for today but what about tomorrow? What about when the company comes up with a promising product which violates one of those ethics? What happens when the companies stock stagnates? What happens when the company is in financial trouble? What happens when the company is coming up with promising new products every day for years that could get it out of that financial trouble? What happens when ethnically minded managers have to argue day in and day out that we should let the company fail rather than dain to implement one of these profitable products which violates just a few of these ethics? Eventually, some combination of desperation and profit motive will break the damn. And often it is a damn that need only break once. Corporate ethics as a whole seems contradictory. The only ethical rules which a company may obey in perpetuity are those that do not for long conflict with profit and those which are backed by effective laws. Even when done with a positive intent, I fear creating positions such as this is begging for eventual corruption.
  • danielheath 6y ago
    She didn’t “refuse to fix it when asked”. She agreed with the proviso that she could have a meeting to figure out what she was and was not permitted to publish. The response was to decline to meet and fire her. It’s entirely plausible that the paper is bunk; however, when someone is willing to come down that hard to prevent an idea being published, I tend to err on the side of “worth finding out what”.
• alangibson 6y ago
  > I understand the concern over Timnit’s resignation from Google - Jeff Dean That's just outright false. She was terminated, effective immediately. Does anyone else find themselves losing a lot of respect for Jeff Dean in all this?
  • sida 6y ago
    She threatened resignation in fairly obnoxious and toxic way. google called her bluff and accepted. No, I haven’t lost respect.
  • deleted 6y ago
    [deleted]
  • amf12 6y ago
    > That's just outright false. She was terminated, effective immediately. Without taking sides (I don't know the full story to understand who is objectively in the wrong here), that's not completely true. She basically gave an ultimatum - meet my demands or I can work on a last date.[1] It is quite common at big tech that once you resign - officially or unofficially - you could be asked to leave effective immediately. Usually happens when you work in critical areas or moving to a competitor or a disgruntled employee. Could this have been better handled? Maybe, but no matter who was in the right or wrong, she wasn't technically fired. [1] https://twitter.com/timnitGebru/status/1334343577044979712 https://twitter.com/timnitGebru/status/1334343577044979712
    • Dylan16807 6y ago
      Threatening to resign is not resigning. Especially as part of a negotiation, and especially when the threat was to "work on" a last date. She was fired.
      • muro 6y ago
        If you threaten to resign, I don't think you should complain if the other side says "ok".
• ardit33 6y ago
  Ok, so basically she is a bullshit social engineer, masquerading as an 'AI ethics researcher' "Gebru’s draft paper points out that the sheer resources required to build and sustain such large AI models means they tend to benefit wealthy organizations, while climate change hits marginalized communities hardest. “It is past time for researchers to prioritize energy efficiency and cost to reduce negative environmental impact and inequitable access to resources,” they write." -- Has she calculated the opposite alternatives? What were the costs of the previous models? How about the alternative of not-having ai-backed search at all. Would worse results be actually worse energy wise? If you are not including the substitues/alternative costs it is just sloppy research, or just done purposely on bad faith. "One is that shifts in language play an important role in social change; the MeToo and Black Lives Matter movements, for example, have tried to establish a new anti-sexist and anti-racist vocabulary. An AI model trained on vast swaths of the internet won’t be attuned to the nuances of this vocabulary and won’t produce or interpret language in line with these new cultural norms." -- Ok, basically she is pissed as AI is picking normal language, that people use, and not using the vocabulary that is in vogue on certain political circles. Basically censure, and forced speech. Why does google even bother at this point? It seems that certain circles just don't want to have positive constructive way, but it is becoming: "my way, or you are X", with "X" being whatever prerogative they will through at you. I think social engineering b.s. should be kept as far away as possible from science. This is turning true ai research into a masquerade to push certain political agendas.
  • _cs2017_ 6y ago
    Do you think the employer weren't aware they were hiring an activist?
    • ardit33 6y ago
      I feel google was being open minded, but the other party was not, and actually being malicious... basically: instead of reform, and fixing whats broken; to tear it down, as it is just another means of oppression.... (yes, that's the mindset of the movement). Her research, looks to be mean spirited, and made to discredit google, so it can be blackmailed into whatever the ideology wants. It is not meant to fix/improve the situation, but used as political amo. Hence it is just a political paper/propaganda, and not proper ai research.
• 0-_-0 6y ago
  "...is known for coauthoring a groundbreaking paper that showed facial recognition to be less accurate at identifying women and people of color, which means its use can end up discriminating against them" Wait, isn't that the other way around? If it can't recognize people of some category then it can't be used to discriminate against them, e.g. the police can't use it to identify peaceful protesters with those characteristics. I wonder what would have happened if the these networks were much better at recognizing women and people of color, would the paper then be about Google designing technology to detect minorities?
  • threatofrain 6y ago
    When police error in identification can mean getting shot, black people deserve to be identified as accurately as individuals of any other race.
    • Dylan16807 6y ago
      That depends enormously on the type of error, so you can't say that as a blanket statement at all.
  • frobozz 6y ago
    If face detection defines them as not a person, then things that rely on there being a person in the field of vision will not work for them. (Like the racist soap dispenser that went viral a few years ago) If face recognition makes the old racist "they all look the same to me" declaration, then peaceful protestors get arrested for looking like criminals.
    • 0-_-0 6y ago
      That's a straw man though, no classification system assigns a "criminal" score. Here's another quote that points out exactly what I was talking about: “There are two ways that this technology can hurt people,” says Raji who worked with Buolamwini and Gebru on Gender Shades. “One way is by not working: by virtue of having higher error rates for people of color, it puts them at greater risk. The second situation is when it does work—where you have the perfect facial recognition system, but it’s easily weaponized against communities to harass them. It’s a separate and connected conversation.” [0] [0] https://www.technologyreview.com/2020/06/12/1003482/amazon-stopped-selling-police-face-recognition-fight/ https://www.technologyreview.com/2020/06/12/1003482/amazon-s...
• darawk 6y ago
  These points are...almost uniformly terrible. > Training large AI models consumes a lot of computer processing power, and hence a lot of electricity. Gebru and her coauthors refer to a 2019 paper from Emma Strubell and her collaborators on the carbon emissions and financial costs of large language models. It found that their energy consumption and carbon footprint have been exploding since 2017, as models have been fed more and more data. You train the model once...and then use it to provide incredibly cheap value for billions of people. Comparing the carbon footprint of a single flight between NYC and LA to the training of a model is...insanely disingenuous. The model gets trained once. The correct comparison would be between the carbon footprint of building the plane. Or, atlernatively, to amortize the carbon footprint of the training over all of the individual queries it answers. > Large language models are also trained on exponentially increasing amounts of text. This means researchers have sought to collect all the data they can from the internet, so there's a risk that racist, sexist, and otherwise abusive language ends up in the training data. This is the only actually legitimate point. This is a real problem, but everyone already knows about this problem, and so if she's going to talk about it she should be doing so in a solutions focused way if she means to contribute anything to the field. She may have done that in the paper, but this review doesn't say so. > The researchers summarize the third challenge as the risk of “misdirected research effort.” Though most AI researchers acknowledge that large language models don’t actually understand language and are merely excellent at manipulating it, Big Tech can make money from models that manipulate language more accurately, so it keeps investing in them. “This research effort brings with it an opportunity cost,” Gebru and her colleagues write. Not as much effort goes into working on AI models that might achieve understanding, or that achieve good results with smaller, more carefully curated datasets (and thus also use less energy). Your criticism is that...tech companies are spending capital on making more profit for themselves? Thats uh, not much of a criticism. Especially when you consider the fact that this technology has positive spillover effects for other groups. These language models can be repurposed to combat racism online, and for all sorts of other things. But even if you ignore that, the premise here is just an utterly trivial near-tautology: "Company invests in things that make it more money". > The final problem with large language models, the researchers say, is that because they’re so good at mimicking real human language, it’s easy to use them to fool people. There have been a few high-profile cases, such as the college student who churned out AI-generated self-help and productivity advice on a blog, which went viral. Sure. You could say this about photoshop, too, and people have. But this technology is going to happen, with or without Google's help. > In his internal email, Dean, the Google AI head, said one reason the paper “didn’t meet our bar” was that it “ignored too much relevant research.” Specifically, he said it didn’t mention more recent work on how to make large language models more energy-efficient and mitigate problems of bias. > However, the six collaborators drew on a wide breadth of scholarship. The paper’s citation list, with 128 references, is notably long. “It's the sort of work that no individual or even pair of authors can pull off,” Bender said. “It really required this collaboration.” Your defense against a claim that specific research was missed is to cite...the length of the citation list? Lol. This argument would hardly pass muster in a forum comment.
• bartread 6y ago
  Good grief - this terrible clickbaity headline writing: "We read the paper that forced Timnit Gebru out of Google. Here’s what it says" The paper DID NOT force her out of Google. Her subsequent behaviour - submitting without approval, rant, ultimatum, and resignation - did. And she wasn't "forced out": she resigned of her own volition. She could have chosen to make improvements to the paper based on the feedback she was given, resubmit it for approval, and then get on with her life, but she went the other way. The headline from the last discussion on Timnit's exit[0] was awful as well: "The withering email that got an ethical AI researcher fired at Google". So bad in fact that it was changed on HN to more accurately reflect what actually happened: "AI researcher Timnit Gebru resigns from Google" (much more neutral and factual in tone). Seriously, what happened to journalistic standards and integrity? Why are the actual events being so forcefully twisted to fit a particular narrative? No wonder the general population struggle to figure out what's true and what's not, and fall victim to all kinds of nonsense, BS theories, and conspiracies. I wish I had a good idea on how to change this behaviour by journalists and publications. (Clearly this is a problem that goes far beyond Timnit's story.) [0] https://news.ycombinator.com/item?id=25292386 https://news.ycombinator.com/item?id=25292386
  • saagarjha 6y ago
    > The paper DID NOT force her out of Google. Her subsequent behaviour - submitting without approval, rant, ultimatum, and resignation - did. And she wasn't "forced out": she resigned of her own volition. She could have chosen to make improvements to the paper based on the feedback she was given, resubmit it for approval, and then get on with her life, but she went the other way. This sounds a lot like taking what Google said without including Timnit’s point of view. Is it really fair to disparage an article for having a title like that when you’ve completely ignored such a large part of the issue?
    • the-dude 6y ago
      Do you dispute her ultimatum and/or offer to resign?
      • 6y ago
• praptak 6y ago
  A company that profits from AI conducts and oversights research of ethics in AI. What could go wrong? It was just a question of time before something like that blew up. Edit/disclaimer: said without judging this particular conflict.
• choeger 6y ago
  While I will not dive into the extremes here, this is yet another battle in the fight over language, right? The most important point of criticism seems to be that the language models are "unethical", i.e., not subject to control by a particular political standard. Am I the only one that finds this line of argumentation highly troubling? The idea that once you do something with language there should be someone proactively controlling you? Should it not always be the output that will be judged by the public?
  • lallysingh 6y ago
    Funny to equate ethics and politics...
    • free_rms 6y ago
      One way to define politics is "difference of opinion over ethics".
  • thu2111 6y ago
    You're not the only one. What we're seeing here is a simple problem spilling out into the public: a lot of AI research is being done at a small number of companies that are in turn dominated by a tiny minority of people with politically extreme views. Those people consider control of language and expression very important to achieve their goals of a racialist and gender-biased society. But they're building AIs which require training on huge language corpuses, and thus inherently learn the way people writing language actually think. Meanwhile the techniques to do "mind control" of AI aren't that good, at least not yet, so this leads to a lot of tensions when AI learns something true but unacceptable to the woke mindset.
  • CydeWeys 6y ago
    I don't agree with this characterization at all. This seems to be a much deeper battle about much more serious potential issues with AI. It's not just about language choice. One of the examples cited in the linked article is how facial recognition wasn't trained on as many examples of lower-albedo people and thus gave more false matches for them, thus potentially leading to false arrests for crimes they didn't commit. This is about a lot more than just language.
• mikaeluman 6y ago
  The problem I find with this has much to do with character. I will not comment on the research (because I do not know the field) but rather comment on why character matters and why motivation is important. Having read her email and tweets; it is quite clear that she is as much a political activist for a far-left “woke” interpretation of the world that I view as of immediate threat to our way of living, democracy and free speech. Of course you can still author great papers but you would be naive to expect her to reach any conclusion that goes against her political goals. In her email she makes this clear. She even posts demands if her paper is not published. In her email she professes to being “dehumanized” for her color and makes it clear that - irrespective of any factors - hiring only 14% women in a year is not acceptable. She continues to say that they had been “enough talking”. We need action. This type of urgency to take action (of course only the actions she herself approves of) and stop talking are clear indications of a person who no longer lives by liberal values but has embraced an ideology to which they now belong. At that point, as an employer of researchers, I would consider her credibility to be severely damaged. If at that point I receive demands from this person “or else”. Then I would of course also accept her resignation. I think the tech industry will need to continue to stand up to the anti-liberal values of these “woke” people before they cause too much harm.
  • akie 6y ago
    > a far-left “woke” interpretation of the world that I view as of immediate threat to our way of living, democracy and free speech. WTF dude. The core argument as to what you say is a "woke" interpretation of the world is to reshape the world in such a way that inherent privilege (for example being rich, being white, being male) does not completely predetermine anyone's outcomes. So that you can still "make it" even if you don't start out as rich, if you're not white, or male. Basically, if you want to boil it down to slogans, it's "the American Dream for everyone". Or, equal opportunities – not equal outcomes. I'm not sure why that is a immediate threat to democracy, free speech, or your way of living.
    • deleted 6y ago
      [deleted]
    • 6y ago
• lsiebert 6y ago
  Ultimately this reads as straight censorship by Google. Or if not by Google, by Google employees, with the tacit approval of their higher ups. They didn't like the viewpoint she expressed, didn't like the criticisms she raised, so they blocked her (well her and 7 co-author's) paper. When she said that was unacceptable and stood her ground, they badmouthed her and fired her. You don't have to agree with the paper's criticisms (and it appears they were just part of a longer paper) to be concerned by viewpoint censorship. If the paper wasn't worthwhile or based on facts, then that would have come out in academic review, either in peer review in the paper, or in subsequent papers rebutting it, or pointing to subsequent changes. That's how academic inquiry works. But if companies can silence ethics researchers who express concerns, whose job, as AI ethicists, is to express concerns, that fundamentally undermines academic inquiry into the topics at hand.
  • curiousgal 6y ago
    > Censorship Man, people really do want to have their cake and eat it too. If you want to publish research freely join an academic research lab, if you value money join an industry lab. You can't have both of those things.
    • lsiebert 6y ago
      She was the technical co-lead of Google's Ethical AI team, publishing papers like this was exactly what she was hired to do. Here's Google's Principles for Artificial Intelligence https://ai.google/principles/ https://ai.google/principles/ From the second paragraph of item 6: "We will work with a range of stakeholders to promote thoughtful leadership in this area, drawing on scientifically rigorous and multidisciplinary approaches. And we will responsibly share AI knowledge by publishing educational materials, best practices, and research that enable more people to develop useful AI applications."
      • silveroriole 6y ago
        If a megafarm hires you to write papers for the Ethical Milk Production team, any modicum of social awareness will tell you that they don’t actually want you to write a paper about the ethics of animal products.
• tsjq 6y ago
  Wow. The pro-google push is really visible in this thread
  • mustuhfa 6y ago
    TBH, I dont think so. I think she is toxic and ample evidence for it. Its normaly for people to side with Google here IMO. Would you want to work with a brilliant jerk?
  • Macha 6y ago
    You can look through my history and realise I've been nothing but critical at Google for their privacy and product lifecycle issues. Not sure I ever weighed in on their anti-union topics but would have been against them there too. The point is, I'm not a Google shill. Still, on this case, on a factual level, the only real dispute is whether this exchange: "Do X or I quit" "Ok, your final paycheck is in the mail and IT will be in touch to organise equipment returns, effective now" Is "accepting a resignation" or "firing". Neither side is disputing that this is how it went down. On an ethical level, again, I'm no fan of Google but Timnit Cebru's previous public actions don't paint her in a good light while Jeff Dean's doesn't have any notable enough to sway my opinion one way or another on his ethical trustworthiness. So based on that, I (and many others) do end up siding with Google. is that a pro-Google push? is someone co-ordinating this? If they are, they haven't contacted me. Don't mistake the fact that Google is often unpopular here with the idea that no Google action can be supported here without interference
    • tsjq 6y ago
      Thanks for the clarity.
• mindrunner 6y ago
  Reading Jeff's email plus her comments on twitter doesn't give the full story. It seems like 1. She did not give them the time required to vet through the paper or followed the processes, plus her email to everyone to stop work on other projects. 2. Google fired her immediately, which might have been different if she wasn't a POC. This
  • viktorcode 6y ago
    If you read the linked material, an ex-Googler calls that a lie based on his experience working as a reviewer of scientific papers there.
• WanderPanda 6y ago
  Thats the kind of problems you get when you let your company turn into a political party...
• ethical-ai 6y ago
  > Though Bender asked us not to publish the paper itself because the authors didn’t want such an early draft circulating online Should not you avoid submitting early drafts which are not good enough to be published yet?
  • dash2 6y ago
    No, it was submitted to a conference, that’s what they’re for. Not the same as submitting for publication.
    • ethical-ai 6y ago
      In AI research conferences are main way to publish, if I understand you correctly. It is generally considered to be a bad practice to submit "such an early draft" for peer review. Moreover, if it was accepted, it would be "circulating online" in conference proceedings. Why is anyone surprised that the draft which is considered not good enough to share with public by authors themselves is banned from sharing (in form of conference publication) by internal review?
      • CydeWeys 6y ago
        There's a lot of speculation going on here. The version of the paper I read did not feel like an early draft at all, but a final one. This is just the excuse Emily Bender is using to avoid providing Technology Review with a copy of the paper. You're right; they wouldn't have submitted an early rough draft for peer review at a conference (these aren't amateurs we're talking about here!). The real reason I suspect she didn't want the paper published is that the other 4 colleagues are still employees at Google and could get in trouble, seeing as how Google doesn't want it published and all.
• hankchinaski 6y ago
  I fail to understand how this is even a story worth covering. Sometime people forget you work FOR someone, who's already paying top dollars for your work. If your interests are not aligned with your employer you certainly have the right to walk away. You surely do not have the right to be seen as a beaten up dog when this was exactly what happens when you ignore feedback, rant and give ultimatums like you count more than anybody else. I am glad she was let go
• tinyhouse 6y ago
  This story blew out of proportion. Social media is sick. Some people's tweets read like she is some kind of a George Floyd level victim. She had the best job one can have. Lead researcher who got resources to build a team to work on her own research interest, having all the freedom. And making a killing doing it. She blew it and her race has nothing to do with what happened. She will be fine, joining academia as a tenured professor most likely. If you care about social justice focus on real victims who really need help. Not about this privileged researcher.
• thinkingemote 6y ago
  Anyone else see the narrative on Twitter to be so much different than Reddit and hacker news? On reddit and hacker news I have never seen any unconditional support for Timmit. Even amongst those anti Google and broadly in support of her, there's no whole agreement with all of what Timmit says; the assertion of her story, the reasons why and the conclusions. that a. She was fired b. That she was fired because of sexism and racism and c. That all research at Google is pointless and should stop until reform. On Twitter there is more vocal support, but here and Reddit even on those most critical of Google and behind Timmit, there's no entire agreement with her. There are comments supporting some parts but none supporting all, none agreeing with the Google Walkout document. Comments pointing out holes with official explanations but not supporting the conclusions. Why is this? I find it odd, I would expect more unequivocal solidarity seen with less public visibility.
  • geofft 6y ago
    Er, who is claiming "all research at Google is pointless and should stop until reform"? Presumably not the person who was trying to do and publish research at Google? (Are you confusing this with her email that said that participating in Google's feel-good diversity and inclusion work, specifically, was pointless and should stop, so that people could focus on their real jobs - such as research - instead of wasting time placating unknown managers?) Anyway, my theory re your question is that Reddit and HN are usually pseudonymous, and so you have no idea whether the person saying something actually knows anything about Google or actually participates in the industry. The norm on Twitter is to use your real identity, and so comments from Googlers and ex-Googlers are boosted there. On Reddit and HN, good-sounding first-principles arguments are boosted. The two models are better at different kinds of discussions.
    • thinkingemote 6y ago
      A bit of a mixup I think. Good to clarify it. The Google walkout blog demands all research needs reform (but no stop) but the email did specify that specific field to stop.
      • 6y ago
• tinyhouse 6y ago
  Last comment about it since I spent too much time on this story. As a researcher in the field I feel bad people don't give Google more credit (I have no affiliation with Google). They created a research environment where researchers have freedom to work on their own interests and publish papers (they publish more than any other company). You don't find many environments like that outside of academia. I still remember Microsoft Research closing down their research lab in the west coast and sending a huge number of researchers home. I can tell you I always apply when there is an opening. So far without luck. If you're a Google Researcher don't forget how lucky you are.
  • tclancy 6y ago
    What? People should check their principles at the door? They literally hired an ethics researcher and then it appears they fired her unethically.
    • mdale 6y ago
      Its a dangerous assumption that a company can conduct ethics research to begin with no? One should just assume it's a highly paid PR group. Academy has it's challenges with ethics and issues of representation in research even without direct influence of share holders and academia supposedly makes huge investments in this area.
    • tinyhouse 6y ago
      Did I say people should check their principles at the door? It's a large research org within a large company. Obv it's not all perfect. I'm sure it has some inefficiencies, politics, etc. It's a company after all and people who accept their offer should know they are going to lose some freedom. But it's still one of the best environments for researchers outside of academia. Google should get some credit for that. Not to mention they get paid extremely well. Researchers in other fields can only dream of such opportunities. Also, it is your opinion it was unethical. I'm sure there's a lot to this story Google cannot share. Nothing in this story to me makes Google Research a place I wouldn't want to be part of.
      • 6y ago
• sokoloff 6y ago
  > An AI model trained on vast swaths of the internet won’t be attuned to the nuances of this vocabulary and won’t produce or interpret language in line with these new cultural norms. As those become pervasive cultural norms, why would the model not adapt to include them?
  • jmeister 6y ago
    They want to be able to control the model before the norms become pervasive. Like how, with activism, you can pressure the NYT into using jargon like “Latinx” when no one outside small progressive circles uses them.
• jpm_sd 6y ago
  Here's why I have absolutely no sympathy for Google in this situation. They hired Gebru as a professional thorn in their side. "Come up in here and be a pain in the ass! Tell us what we're doing wrong!", they said. "We're Enlightened Corporate America, after all!" She is a chess piece in the game of Wokeness Street Cred. She then proceeded to do the job she was hired for, and now they're all "Hey lady, Here At Google that's not how we do things". That said, as a manager, I would have "accepted her resignation" and/or fired her without hesitation.
  • ghaff 6y ago
    I don't really have a perspective on this particular case. But yeah. "Companies" (really senior people within those companies) hire people who are known in their field, bring an outside perspective, have an independent voice, etc. And then those people (or perhaps different senior people) get unhappy when the individual who was hired doesn't go through channels or otherwise toe the party line. Either both sides find a middle ground by some combination of putting certain opinions in a box and choosing to ignore the use of channels or messages outside of official guidelines--or you part ways. I have some experience with this. Not to nearly the same degree. But I had something of a tiff with an exec my second day on the job in a prior role.
  • Wowfunhappy 6y ago
    I'm less cynical about this as a concept. Would you feel differently if, say, Apple hired a "Privacy Watchdog", with a long history of activism on the topic? Someone you could trust to speak out if something was amiss. If the person is later fired, that's a sign that something is wrong at the company. But if they stay, and have generally good things to say, that's a sign the company can be trusted. I do think this is a good system!
    • ghaff 6y ago
      It's a difficult type of role. See the NYT getting rid of its public editor/ombudsman.[1] I'd actually have trouble seeing Apple hiring someone like that given how tightly they control external messaging/speaking/etc. It's really hard for execs, including but not limited to PR, to have worked really hard on getting some message/narrative accepted by the media/analysts/etc. and then have a high profile employee on social media or in a major publication saying it's sort of B.S. even as an occasional thing. ADDED: It doesn't make it "right" for them to storm into their boss' office and demand said person be fired but it is at least an understandable human reaction. I think anyone who takes that kind of role has to understand they'll be a lightning rod, feel passionate about it anyway, and understand they may well eventually set off a tripwire and be fired in a very public and humiliating way. [1] https://www.thedailybeast.com/why-the-new-york-times-fired-its-public-editor-in-favor-of-a-reader-center https://www.thedailybeast.com/why-the-new-york-times-fired-i...
• eecks 6y ago
  AI ethics research is funny. It's obviously important but it's also kind of ... obvious. I am surprised they get paid so much. * Lots of processing uses more energy.. * Large amounts of data might contain bad data (garbage in, garbage out) * Wealthy countries/communities have the most 'content' online so less wealthy countries/communities will be unrepresented. here's some more: * AI being forced to chose between two "bad" scenarios will result in an unfavorable outcome for one party * AI could reveal truths people don't want to hear e.g. it might say the best team for a project is an all white male team between 25 - 30 rather than a more diverse team. It might say that a 'perfect' society needs a homogeneous population. * AI could disrupt a lot of lower paid jobs first without governments having proper supports and retraining structures in place
  • ur-whale 6y ago
    >AI ethics research is funny. Calling "AI ethics" research makes it feel like it's actual science. It's in practice much closer to "social science" than actual science. And just like "social science" it isn't funny at all but rather a giant hoax.
    • flybrand 6y ago
      It almost reads like ‘Ethics Philosophy applied to AI’
    • andybak 6y ago
      As much as I agree social "science" is often flawed, your flippant dismissal of an entire field with an extensive pedigree strikes me as unlikely to come from a place of knowledge and familiarity.
    • UncleMeat 6y ago
      She's got a PhD in CS from Stanford. She is published in CS conferences. Even if you want to shit on the social sciences for some reason, it is clear that this work is computer science.
    • oblio 6y ago
      The most important problems in life are ill-defined. We still need to work at solving them, with the limited means we have. If we'd only work with hard sciences, life would never get better. Many things in our lives have advanced based on what could be considered guesses (for example capitalism doesn't really have a rigorously defined mathematical model, yes, they are some hand-wavy explanations but nothing you'd put in a solid math or physics paper).
• ur-whale 6y ago
  It's not the paper that forced her out: she forced herself out.
  • TotallyHaram 6y ago
    Yes, by trying to use extortion against her employer to advance her own political agenda. Highly unethical. I am glad that she will not be participating in 'ethics' and machine learning any longer because it was totally ironic that someone this unethical would even be considered for the position.
• mlthoughts2018 6y ago
  The summarized sections of the paper seem very, very weak to me. 1. Environmental footprint of technology must always be considered as a trade-off for what you get in return. Why do we spend energy on a giant render farm for Pixar? Because the cinematic artwork is worth that environmental cost for many people. Obviously we should pursue improvements in environmental outcomes, but that is not a goal unto itself in a vacuum apart from all other side effects of a technology. Is it worth ~5 car-lifetimes to train GPT? I would say overwhelmingly yes. It reminds me of an anecdote from The Beginning of Infinity by David Deutsch, where some ethicists argued about whether it was a useful human endeavor to invent color TV monitors back when they first hit the market. Why would you need to spend resources creating something besides black and white TV? Yet today and for decades, color monitors are used as critical tools to diagnose diseases and save lives. 2. Nobody is required to accept “wokeness” vocabulary, and indeed one of the signs of a crank or a quack is making up a fiefdom of new vocabulary and collecting rent in the form of social authority for the validity and requirement of that new vocab. Nobody is required to be on the cutting edge of how activist language changes, and it seems like a disingenuous way to try to make a cottage industry out of one’s own expertise in rapid changes to activist language. As long researchers are stating what corpus is used, and making tools available that allow peer reviewers to audit that corpus, they are meeting their obligations to their professional field and to society. We should be happy that language researchers would produce lots of papers and lots of models on many sets of corpora, and as the cost of training these models gets cheaper, and the cost of curating the corpora gets cheaper, we can expect to see better variety of curated large corpora, domain-specific corpora and other things. 3. Researcher opportunity cost is perhaps the most ridiculous objection. Researchers are free agents to decide what they want to study. In most PhD programs, especially in machine learning, the project selection is entirely up to the student. If Timnit wants there to be different research priorities, well, news flash, but she is only one of eight billion. Unless she wants to raise money to give as research grants that tie the researchers to her pre-decided methods of inquiry, she really has nothing to say here. 4. Inscrutable models - this is the only one where there is any point. If the models can produce harmful outcomes and they are inscrutable, then debugging or safeguarding them is a real problem. But this has been true for almost all types of computer science algorithms. Of course we should work on methods that synthesize clarity from the prediction mechanism of large neural nets, but that is also not a criticism of neural nets. That’s just a need for more technology. Overall the main points of this paper seem full of themselves, arrogant and overly self-important, especially with wildly ridiculous connections to “wokeness” vocabulary. Given the immediate nuclear option of the ultimatum and email that Timnit sent, I suspect the full text of the paper would be even more unacceptable. Google has plenty of bullshit issues with the way it treats employees and transparency of decision making. Rejecting this publication approval was not one of them.
• paganel 6y ago
  Of course that Jeff Dean is white, male and I presume hetero, coming from (I also presume) at least a middle-class family, there's no chance he will understand the fact that > AI-generated language will be homogenized, reflecting the practices of the richest countries and communities. because this hasn't ever had any negative effect on him or on the people which he represents. Bonus points for him being "revered" for being an excellent engineer (which I bet he his), so small chances of any humility coming back from him. If this looks like a "calling out names" post that's because it is such a post, people who hold power (in this case Dean) in this type of situations and who use it contrary to society's needs (which demands more inclusiveness, less social friction) need to be called out.
  • FartyMcFarter 6y ago
    If it's OK to assume that Jeff Dean cannot understand certain things because of his ethnicity, is it also OK to assume that people of other ethnicities cannot do or understand certain things? Is this the kind of culture we should be encouraging?
    • paganel 6y ago
      > other ethnicities cannot do or understand certain things? I I don't think that those ethnicities have been in the same position of power as the one represented by Dean, so your point is moot.
      • deleted 6y ago
        [deleted]
  • igravious 6y ago
    > I presume hetero ? > coming from (I also presume) at least a middle-class family ? That's quite a lot of presumptions. I note in passing the asymmetry of the use of homo and hetero – the former is offensive, the latter is informal, interesting no? How'd that come about? –– [1] https://dictionary.cambridge.org/dictionary/english/homo https://dictionary.cambridge.org/dictionary/english/homo [2] https://dictionary.cambridge.org/dictionary/english/hetero https://dictionary.cambridge.org/dictionary/english/hetero
• nashalo 6y ago
  Not totally on topic, but on threads about researchers, I see this trend that when we're talking about a male researcher we only mention the family name but when talking about a female researcher we're using mostly the first name. Case in point, at time of writing there are 26 mentions of "Timnit" vs 16 mentions of "Gebru" on this thread. I don't think there are bad intentions behind this, but it really comes off as infantilizing so maybe we'd be better off calling her "Gebru"?
  • spunker540 6y ago
    I’d love to see the HN discussion on a study titled “An analysis of unconscious bias on HN”! Great point though
  • machinelabo 6y ago
    There could be many other reasons: - Outside media continues to refer to her as Timnit including NYTimes which affects HN users - If someone’s name is Gorot Trzebiatowski (please forgive me if you have such a name), most people would go with Gorot because it is shorter and easier to pronounce / write. Subconsciously, these things happen. You’re right - No need to assume malice where probably there isn’t. This is conspiratorial thinking - find some pattern of data that supports mainstream narrative and make conclusions from it. This is what QAnon conspirators do all day. Although I’m curious about your motivation for analyzing this :-)
  • web007 6y ago
    I haven't noticed this elsewhere, but haven't been paying attention. It'll be interesting to see how much I notice it henceforth - interesting and probably disappointing. Regarding first/last name confusion, how many people here on HN _know_ which is her first name and which is her last? It's a lot easier for Western Anglicized speakers to get it wrong with non-Western non-Anglicized names than with "Jeff Dean", whether intentionally or not.
  • 4bpp 6y ago
    Ethiopian names seem to generally not follow the western first name/surname pattern, instead using the Semitic system where you list the male lineage (cf. https://ethnomed.org/culture/ethiopian/ https://ethnomed.org/culture/ethiopian/). I remember this information receiving some unusually mainstream exposure thanks to the moments in the limelight that https://en.wikipedia.org/wiki/Tedros_Adhanom https://en.wikipedia.org/wiki/Tedros_Adhanom enjoyed this year. (Now, I don't actually have enough of a sense of Ethiopian ethnicities to be confident as to whether this holds true for all of them and whether Timnit Gebru belongs to one for which it does, but the case of @DrTedros certainly left me with a general heuristic saying that referring to her as Gebru may be an ignorant foreigner move whereas using the "first" name (as US academics generally refer to each other anyway) is almost certainly safe.)
• vmception 6y ago
  Googlers are waaay too comfortable with their message board system and if they stay there long enough and keep interacting with it they’ll get burned
• auggierose 6y ago
  Doesn't make the bar for Google publishing quality standards? That is the most ridiculous thing I have ever heard.
• pabl0rg 6y ago
  Maybe the real problem with the paper is that it points out that GPT-3 and the like can completely distort search results by filling the web with auto-generated spam. Botnets could link spam pages to each other and even generate traffic. If that got into the wrong hands, we would be unable to distinguish truth from fiction. If the general public heard of this, Google’s stock price might be hurt. In such a bot-filled world, humans might prefer to start their searches in walled-gardens or on sites that could better-validate content.
• ioctrl 6y ago
  As a white man, when my work is rejected, I assume it's because my work needs improvement. This is an advantage of being a white man. When things don't go my way, I don't need to wonder if it's a macro-aggression, an act of systemic discrimination, or suppression of a marginalized voice. I don't envy my colleagues who navigate these waters. Furthermore, even if my manager or employer makes it an explicit priority to promote more women or PoC, I don't question whether my own gender or skin color will ultimately work against me at promotion time. I grew up in a time and environment where I didn't need to question that, so I tend not think about it. Even if I did think about it, it's a culturally inappropriate question to ask, so my only option is to keep my head down and work harder.
  • nxmnxm99 6y ago
    As a minority growing up in a developing world with actual problems, I never once even heard of these "waters" until immigrating to the most wealthy & powerful country in the world. Such is the product of a country which hasn't had any real problems in half a century.
• saeranv 6y ago
  I feel very disheartened that whatever good Timnit Gebru is doing in her work seems to be undermined by her constant antagonism and hostility. Based on this article, and additional context from other Google employees, I think she is in the right, but that she herself is such an obnoxious person it's hard for a lot of us here to be able to separate the issue with her personality. My first introduction to Gebru (and probably a lot of others here) was through her fight with Yann LeCun. Whatever everyone thinks about that debate, one thing was clear: LeCun consistently asked his supporters to not attack Gebru, and tried to engage the discourse, whereas she repeatedly encouraged her supporters to attack LeCun, and avoided his points. FWIW, I myself am a minority. I am very sympathetic to the investigation of AI ethics and Gebru's general area of work. I also completely understand and support the need for activists to disrupt norms: we wouldn't have civil liberties if activists weren't willing to disrupt middle-class norms. But I think for exactly that reason activists have a responsibility (a greater responsibility even) to be selective about which norms or civilities they choose to disrupt. They do such enormous harm when their actions can be picked apart and dismissed for completely unrelated issues - in this case because Timnit doesn't seem to engage in good faith discussions with other experts in her field. It sucks, I really hate that this is what ethics in tech is going to be associated with.
  • joshuamorton 6y ago
    > whereas she repeatedly encouraged her supporters to attack LeCun, and avoided his points. This is historical revisionism, colored by your misremembering of the events. If you go back and look, you'll be unable to find Gebru encouraging anyone to attack LeCun. Nor will you find LeCun making an effort to engage with Gebrus research. He never so much as acknowledges the existence of her publications when asked to read or comment on them. E: as an amusing example of the importance of this kind of research, my phone decided that when I typed "Gebru", I meant "Henry".
    • saeranv 6y ago
      So as I recall, you're right that she didn't encourage people explicitly, but she kept retweeting other supporters attacking LeCun, whereas LeCun kept telling his supporters to not attack her, and talked very positively of her. I agree my phrasing is muddying the waters, I should have been more precise. On another note, it's interesting to me that I keep seeing the LeCun debate come up here again and again as an example of her being a bad-faith debater, given that it technically doesn't have anything to do with this Google debate. Whether we agree with her on the substance, I feel like it strengthens my point that she badly damaged her perception amongst the ML/AI crowd, and their willingness to give her the benefit of the doubt in the way she chose to debate LeCun.
• mensetmanusman 6y ago
  The lifetime output of 5 cars is nothing in a nation of over 300,000,000. If AI can manage peak energy usage efficiently and improve the grid by 1% it will have made a positive impact over 100 fold.
• ipsocannibal 6y ago
  I'm curious if the paper posits any solutions to the risks highlighted? There was mention of references to work on making model training more energy efficient but that was it.
• nickdothutton 6y ago
  Kicking out (or having resign) an AI researcher is a lot less costly in PR terms than having a large scale AI fail like the one Google experienced when their photo software started identifying black people as monkeys in 2015. Gebru is a fuse that just went pop, and can now be replaced with a new one quite easily.