10 ms·

Microsoft’s Kate Crawford: ‘AI is neither artificial nor intelligent’

156 points by 9wzYQbTYsAIc 5y ago

arcanus 5y ago
"[Microsoft] Unusually, over its 30-year history, it has hired social scientists to look critically at how technologies are being built. Being on the inside, we are often able to see downsides early before systems are widely deployed. My book did not go through any pre-publication review – Microsoft Research does not require that – and my lab leaders support asking hard questions, even if the answers involve a critical assessment of current technological practices." Interesting to note she opens with a veiled attack at Google's internal review practices. For those not aware: https://www.google.com/amp/s/www.theverge.com/platform/amp/2021/4/13/22370158/google-ai-ethics-timnit-gebru-margaret-mitchell-firing-reputation https://www.google.com/amp/s/www.theverge.com/platform/amp/2...
- stephc_int13 5y ago
  No so veiled, and fully warranted, IMHO.
- ocdtrekkie 5y ago
  My first thought was that even giving this brief interview would be a fireable offense for Google's AI Ethics team members.
- antonzabirko 5y ago
  Good, glad google is facing at least some sort of consequences as minor as they might be. It might even end up snowballing against them.
- sanity31415 5y ago
  It sucks when facts get in the way of a fashionable narrative, but the Google AI researcher was fired for demanding the names of an internal review panel who rejected her paper. She had a reputation for accusing her colleagues of bigotry and other toxic behavior. The r/machinelearning thread provides a more balanced perspective: https://www.reddit.com/r/MachineLearning/comments/k77sxz/d_timnit_gebru_and_google_megathread/ https://www.reddit.com/r/MachineLearning/comments/k77sxz/d_t...
  - joshuamorton 5y ago
    This needn't be relitigated, but you're at least misleading, if not factually wrong. The "internal review panel" wasn't at least at the time, known to be a real process. In other words, the sequence of events was that the paper was written and approved via normal channels. After this, the authors were informed that they needed to withdraw it. Confused, they asked for more information (by what process was this decided, and by who, was it peers, legal, an executive who wanted to bury the paper, etc.) and were stonewalled. To my knowledge these questions still haven't really been answered. I think its very reasonable request to want to know who or what caused your paper to get blocked. In normal peer review, even when it is blind, you can work with the reviewers to update the paper and see their feedback. These opportunities weren't readily provided.
    - sanity31415 5y ago
      So it's normal to demand the identities of a blind review panel? What exactly did I say that was "factually wrong" as you claim?
      - joshuamorton 5y ago
        > So it's normal to demand the identities of a blind review panel? It seems reasonable to ask "wait what blind review panel" when one appears out of thin air.
        sanity31415 5y ago
        She did a lot more than that. Here is the email that got her fired: Hi friends, I had stopped writing here as you may know, after all the micro and macro aggressions and harassments I received after posting my stories here (and then of course it started being moderated). Recently however, I was contributing to a document that Katherine and Daphne were writing where they were dismayed by the fact that after all this talk, this org seems to have hired 14% or so women this year. Samy has hired 39% from what I understand but he has zero incentive to do this. What I want to say is stop writing your documents because it doesn’t make a difference. The DEI OKRs that we don’t know where they come from (and are never met anyways), the random discussions, the “we need more mentorship” rather than “we need to stop the toxic environments that hinder us from progressing” the constant fighting and education at your cost, they don’t matter. Because there is zero accountability. There is no incentive to hire 39% women: your life gets worse when you start advocating for underrepresented people, you start making the other leaders upset when they don’t want to give you good ratings during calibration. There is no way more documents or more conversations will achieve anything. We just had a Black research all hands with such an emotional show of exasperation. Do you know what happened since? Silencing in the most fundamental way possible. Have you ever heard of someone getting “feedback” on a paper through a privileged and confidential document to HR? Does that sound like a standard procedure to you or does it just happen to people like me who are constantly dehumanized? Imagine this: You’ve sent a paper for feedback to 30+ researchers, you’re awaiting feedback from PR & Policy who you gave a heads up before you even wrote the work saying “we’re thinking of doing this”, working on a revision plan figuring out how to address different feedback from people, haven’t heard from PR & Policy besides them asking you for updates (in 2 months). A week before you go out on vacation, you see a meeting pop up at 4:30pm PST on your calendar (this popped up at around 2pm). No one would tell you what the meeting was about in advance. Then in that meeting your manager’s manager tells you “it has been decided” that you need to retract this paper by next week, Nov. 27, the week when almost everyone would be out (and a date which has nothing to do with the conference process). You are not worth having any conversations about this, since you are not someone whose humanity (let alone expertise recognized by journalists, governments, scientists, civic organizations such as the electronic frontiers foundation etc) is acknowledged or valued in this company. Then, you ask for more information. What specific feedback exists? Who is it coming from? Why now? Why not before? Can you go back and forth with anyone? Can you understand what exactly is problematic and what can be changed? And you are told after a while, that your manager can read you a privileged and confidential document and you’re not supposed to even know who contributed to this document, who wrote this feedback, what process was followed or anything. You write a detailed document discussing whatever pieces of feedback you can find, asking for questions and clarifications, and it is completely ignored. And you’re met with, once again, an order to retract the paper with no engagement whatsoever. Then you try to engage in a conversation about how this is not acceptable and people start doing the opposite of any sort of self reflection—trying to find scapegoats to blame. Silencing marginalized voices like this is the opposite of the NAUWU principles which we discussed. And doing this in the context of “responsible AI” adds so much salt to the wounds. I understand that the only things that mean anything at Google are levels, I’ve seen how my expertise has been completely dismissed. But now there’s an additional layer saying any privileged person can decide that they don’t want your paper out with zero conversation. So you’re blocked from adding your voice to the research community—your work which you do on top of the other marginalization you face here. I’m always amazed at how people can continue to do thing after thing like this and then turn around and ask me for some sort of extra DEI work or input. This happened to me last year. I was in the middle of a potential lawsuit for which Kat Herller and I hired feminist lawyers who threatened to sue Google (which is when they backed off--before that Google lawyers were prepared to throw us under the bus and our leaders were following as instructed) and the next day I get some random “impact award.” Pure gaslighting. So if you would like to change things, I suggest focusing on leadership accountability and thinking through what types of pressures can also be applied from the outside. For instance, I believe that the Congressional Black Caucus is the entity that started forcing tech companies to report their diversity numbers. Writing more documents and saying things over and over again will tire you out but no one will listen. Timnit
- Aunche 5y ago
  >Being on the inside, we are often able to see downsides early before systems are widely deployed. Well they failed to catch that Microsoft Tae would be instantly turned into a internet Nazi bot.
rektide 5y ago
James Martin called it true in 2001 when he described the coming machine learned systems Alien Intelligences. unintelligible inhuman systems. from After the Internet: Alien Intelligence.
stephc_int13 5y ago
I agree with most of the content of this article. Especially this part: "Ethics are necessary, but not sufficient. More helpful are questions such as, who benefits and who is harmed by this AI system? And does it put power in the hands of the already powerful?" Distribution of power is the most important political question, more important than distribution of wealth or what we call ethics, that is always biased and hard to measure.
- dec0dedab0de 5y ago
  I agree, but wouldn't those questions, or atleast any opinions on the answers to those questions, just be part of ethics?
- spoonjim 5y ago
  But anything that benefits Microsoft, one of the most powerful companies in the world, is necessarily putting more power in the hands of the already powerful.
ThalesX 5y ago
> But from the beginning there was pushback and more recent work shows there is no reliable correlation between expressions on the face and what we are actually feeling. And yet we have tech companies saying emotions can be extracted simply by looking at video of people’s faces. We’re even seeing it built into car software systems. I've been involved with a startup where the CEO was certain we're only a step away from replacing humans in recruiting healthcare workers with AI analyzing expressions in self-submitted interview videos, as well as analysis of quizzes. I got pushed aside from that startup for being the 'obnixous technical humanist' (which is a caracterization I wear proudly). > AI is neither artificial nor intelligent. It is made from natural resources and it is people who are performing the tasks to make the systems appear autonomous. I've argued as a technological consultant times and times again that the effort they are spending on automating humans out, would be better spent in making those humans happier and augumenting them with an automated system. The fact that the CEO saw his low level employees as just temporary assets ready to be replaced meant that their life in the company was miserable. > This April, the EU produced the first draft omnibus regulations for AI I digress. I'm wondering what they're plan of action is since it was a European startup and since April we're theoretically pushed away from using AI to gauge candidates and all of their VC investment money came with the promise of 'automating healthcare recruitment and scaling globally'.
rgbrenner 5y ago
AI is neither artificial nor intelligent. It is made from natural resources and it is people who are performing the tasks to make the systems appear autonomous. Making something from natural resources does not make it natural. If that were the case, we wouldn't have the word "artificial", since everything we make comes from natural elements and everything would be "natural". The fact that you took natural resources and built something else from it--that it didn't exist in nature already--makes it artificial. AI is definitely artificial.
- pkdpic_y9k 5y ago
  That's a nice clean breakdown of that wording issue. I get her point though I guess, but I was hoping there might be something deeper.
- deleted 5y ago
  [deleted]
- jcims 5y ago
  I had the exact same reaction, but I wonder if I'm too close to it to appreciate what the term might conjure up in the broader (voting) public that isn't in the industry. (FWIW I kind of like the term synthetic intelligence as an alternative.) I do agree wholeheartedly that, much as in the case of cryptocurrency, there isn't an intuitive link between the product and the resources it requires to create and operate. The fact that GPT-3 would cost millions to reproduce in the commercial market doesn't even compute for me.
- efnx 5y ago
  I’m not arguing with your point, but let me offer another thought. The crux of this problem as I see it is that people consider themselves as separate from nature. The truth is that we are part of nature and the things we make are still part of nature. The opposite of “natural” is not “artificial” or “man-made” - it’s “supernatural”! Of course the word “artificial” is useful to classify things for our safety and benefit, but we are not supernatural and so the things we create still exist in nature - like we are the hand of the universe reconfiguring itself. This artificial separation of the human from nature has been popular in the past and I hope we overcome it.
smoldesu 5y ago
Honestly the hardest part about integrating AI into our daily lives is the fact that we don't really have AI yet. We have made great advances in the fields of machine learning and neural networks, but actually getting a computer to make educated decisions is hard. The current issue is that all of these models are black boxes: an ML model can guess which data will come out, but it can't really fully understand what it knows. It can identify slightly harder to notice patterns, but it can't actually think. We adopted the phrase "AI" far too early in the field, and I suspect there will be another few decades before we have the technical and scientific capability to make real artifical intelligence a thing.
- otabdeveloper4 5y ago
  > The current issue is that all of these models are black boxes Not quite. It'd be great if that was the only issue with neural network ML. A much bigger issue is that neural networks have extremely limited applicability. They're great for classification problems when you can have huge training datasets, but for many common problems they're useless. One obvious class of problems is time series prediction - extremely important in life and for business, and something neural networks are no good for.
  - throwaway3699 5y ago
    Doesn't transfer learning level the playing field a bit? We already have excellent, off-the-shelf models for NLP, computer vision, ODT, etc... that just need fine-tuning for a particular business or domain problem. I think the 'huge training datasets' requirement is lessening every day. If you're doing something novel, then sure. I can see that.
    - otabdeveloper4 5y ago
      Predicting time series isn't "novel". It's the oldest data science problem in the world. And ML can't solve it yet.
      - 5y ago
jjcon 5y ago
> AI is neither artificial nor intelligent. It is made from natural resources and it is people who are performing the tasks to make the systems appear autonomous. People performing the tasks? As in coding tasks? That’s kinda the definition of artificial. If AI spawned naturally… that’s what wouldn’t be artificial. Or is she referring to data labeling and forgetting the many non supervised areas of AI? (Not that labeling would make it less artificial) Or is she suggesting that anything made from natural resources is natural… is anything not made from natural resources? The reason for artificial in artificial intelligence is to juxtapose against biological intelligence not speak to resource usage. I think arguing on the ‘not intelligent’ front is fine but that’s just kind of a word game. The field seeks intelligence and is fine making stupid ai if it means better ai later. Unless she is getting into notions of a soul or some bs about consciousness in which case we have left the realm of science altogether. Any way you slice it seems more like an inane point for making a headline than one for substantive discussion.
- Cybotron5000 5y ago
  It seems to me somewhat similar to an argument put forward by Jaron Lanier: that what is termed artificial intelligence’s (or ‘deep learning’’s etc.) current successes, at least as far as something like the translation services of the big corporations for example, is actually built by leveraging corpuses that are the fruits of a huge amount of individual human effort. Sometimes, say with the ‘Mechanical Turk’ thing that Amazon has, or CAPTCHAs or something, this is a more explicit connection (…bit Wizard of Oz! :) As I understand it, he proposes that an alternative to UBI etc. might be micro-compensations (or transactions) in return for providing this data. This might be a prelude/transition to a stage where our basic needs (on a sort of Mazlow’s hierarchy) were met by A.I., or that there might be increasingly creative or interesting ways to complete some tasks that we could go on refining forever.
  - jjcon 5y ago
    > are the fruits of a huge amount of individual efforts That doesn’t make it any less artificial does it? If anything that makes it more so. That is also again only speaking to supervised learning (and only the fraction of it where datasets are curated not collected) not AI in general
mjburgess 5y ago
> Time and again, we see these systems producing errors ... and the response has been: “We just need more data.” but... you start to see forms of discrimination... in how they are built and trained to see the world. Thank goodness this perspective is getting out there. I have recently been incensed by the opposite view, that the bias is within "the data" only: https://www.youtube.com/watch?v=6jbin15-TcY https://www.youtube.com/watch?v=6jbin15-TcY . This is wholly false. Machines which analyse the world (ie., actual physical stuff, eg., people) in terms of statistical co-occurances within datasets cannot acquire the relevant understanding of the world. Consider NLP. It is likely that an NLP system analysing volumes of work on minority political causes will associate minority identifiers (eg., "black") with negative terms ("oppressed", "hostile", "against", "antagonistic"), etc. And thereby introduce an association which is not present within the text. This is because conceptual association is not statistical association. In such texts the conceptual association is "standing for", "opposing", "suffering from", "in need of". Not "likely to occur with". There are entire fields sold on a false equivocation between conceptual and statistical association. This equivocation generates novel unethical systems. AI systems are not mere symptoms of their data. They are unable, by design, to understand the data; and repeat it as-if it were a mere symptom of wordly co-occurance.
- throwaway3699 5y ago
  I don't know, inference from data is literally how all decisions are fundamentally made. Why wouldn't it be possible to create models that learn this particular pattern?
  - mjburgess 5y ago
    Here's where the words "data", "pattern", etc. become unhelpful. Learning, as in what we do, is not learning associations in datasets. It is learning "associations" between: our body state and the world as we act. It's a sort of: (action, world, body, sensation, prior conceptualisation, ...) association. (Even then, our bodies grow and this is really not a formal process.) This is, at least, what is necessary to understand what words mean. Words are just tools that we use to coordinate with each other in a shared (physical) world. You really have to be here, with us, to understand them. Words mean what we do with them. Meaning has a "useful side-effect". It turns out when we are using words their sequencing reveals, on average, some commonalities in their use. Eg., when asking "Can you pass me the salt?" I may go on to ask, "and now the pepper". And thus there is a statistical association between the terms "salt" and "pepper". But a machine processing only those associations is completely unaware there is anything "salt" to pass, or even that there are objects in the world, or people, or anything. Really, the machine has no connection between its interior and the world, the very connection we have when we use words. When a machine generates the text "pass me the salt" it doesnt mean it. It cannot. There is no salt it's talking about. It doesnt even know what salt is. A machine used to make decisions concerning people, unaware of what a person even is, produces new unethical forms of action. Not merely just "being racist because the data is".
TOSSAWAY_1 5y ago
Lol, ai is racist Therefore the people making ai are racist. Smart. I'll weaponize accordingly.
ltbarcly3 5y ago
Ahh so it is naturally occurring without human artifice non-intelligence. Got it. NOWHANI
eunos 5y ago
It should not matter whether it is artificial or intelligence or neither, as long as it get the jobs done.
- simonswords82 5y ago
  Except when people are using the term AI to sell nonsense that doesn't do what it says on the tin.
version_five 5y ago
> ImageNet has now removed many of the obviously problematic people categories – certainly an improvement – however, the problem persists because these training sets still circulate on torrent sites [where files are shared between peers]. This is the scariest part of the article. The idea that some central authority should be censoring and revising datasets to keep up with political orthodoxy, and we should be rooting out unauthorized torrent sharing of unapproved training data. From a technical point of view, the common reason we pre-traini on imagenet is as the starting point for fine tuning for a specific use case. The diversity and size of the dataset makes good generic feature extractors. If you're using a ML model to identify people as kleptomaniac or drug dealer or other "problematic" labels, you're working on some kind of phrenology and it doesnt take an "AI ethicist" to know you shouldn't do it. But that's not the same as pretraining on imagenet, and certainly doesn't support trying to make datasets align with today's political orthodoxy.
Husafan 5y ago
Discuss amongst yourselves.
t_von_doom 5y ago
> The idea that you can see from somebody’s face what they are feeling is deeply flawed. I don’t think that’s possible. I agree with most of the article but this point I disagree with. Non verbal communication is a huge component in how we interact with each other. Covid example: When on calls where no cameras are on I get far less feedback when presenting to a 'room' compared to if I was to present in person or have cams on, even if the room stay silent in both situations. By looking at faces I can see who is distracted, who looks confused and whether what I am saying is being received well or poorly. Did that joke get smiles (polite or genuine ones?) or eye rolls? Now - do I think the current SOTA algorithms are at this level of nuance? No, definitely not. But to say it isn't at all possible is ridiculous in my opinion
- cocoafleck 5y ago
  I believe that people (usually) choose to communicate with faces, but just as people can lie with words they can lie with faces.
slver 5y ago
Even in situations where there's an objective reason for AI performing poorly with certain groups (like the worse light contrast on facial details with many photos of dark-skinned people), we go back to default and seek to cast moral judgment on someone for being racist, sexist, or something of the sort. Because you can't blame AI, we blame the people programming it, collecting data for it and training it. It seems we're trying to prevent a full-on moral panic that would be caused by the realization that "discrimination" is an objective need by the objective nature of the problem in some cases. For example there are about 14% black people in the US. If you have a FAIR SET OF DATA to train from, 14% of those faces would be black. No, they're not underrepresented, they're literally accurately represented. But this means the AI will be worse with those people. So what do we do? If we artificially up the "quota" and train with 50% black faces, the AI will now underperform with non-black faces. So then you need to train two networks: black faces and non-black faces, and then you need to "discriminate" between both, and pick the right network depending on the race you're working with. Math is racist sometimes.
- throwaway3699 5y ago
  Race aside, you can make this argument for any social strata. Gender, race, nationality, social class, etc... I don't believe most models will perform very different with a balanced training set.
yarg 5y ago
Yes it is, and of course it's not - it's artificial. If it were intelligent we'd be calling it "synthetic".
mark_l_watson 5y ago
I am reading her book and so far, I am enjoying it. The book really made me think of externalities for tech that I use and earn a living with. Costs of these externalities are environmental and human suffering. Recommended.
andyxor 5y ago
oh God, not again about "ethical AI", Gebru was fired for being a potz, who would want an employee throwing ultimatums and doxxing colleagues on social media

156 points by 9wzYQbTYsAIc 5y ago

"[Microsoft] Unusually, over its 30-year history, it has hired social scientists to look critically at how technologies are being built. Being on the inside, we are often able to see downsides early before systems are widely deployed. My book did not go through any pre-publication review – Microsoft Research does not require that – and my lab leaders support asking hard questions, even if the answers involve a critical assessment of current technological practices." Interesting to note she opens with a veiled attack at Google's internal review practices. For those not aware: https://www.google.com/amp/s/www.theverge.com/platform/amp/2021/4/13/22370158/google-ai-ethics-timnit-gebru-margaret-mitchell-firing-reputation https://www.google.com/amp/s/www.theverge.com/platform/amp/2...

stephc_int13 5y ago
No so veiled, and fully warranted, IMHO.
ocdtrekkie 5y ago
My first thought was that even giving this brief interview would be a fireable offense for Google's AI Ethics team members.
antonzabirko 5y ago
Good, glad google is facing at least some sort of consequences as minor as they might be. It might even end up snowballing against them.
sanity31415 5y ago
It sucks when facts get in the way of a fashionable narrative, but the Google AI researcher was fired for demanding the names of an internal review panel who rejected her paper. She had a reputation for accusing her colleagues of bigotry and other toxic behavior. The r/machinelearning thread provides a more balanced perspective: https://www.reddit.com/r/MachineLearning/comments/k77sxz/d_timnit_gebru_and_google_megathread/ https://www.reddit.com/r/MachineLearning/comments/k77sxz/d_t...
- joshuamorton 5y ago
  This needn't be relitigated, but you're at least misleading, if not factually wrong. The "internal review panel" wasn't at least at the time, known to be a real process. In other words, the sequence of events was that the paper was written and approved via normal channels. After this, the authors were informed that they needed to withdraw it. Confused, they asked for more information (by what process was this decided, and by who, was it peers, legal, an executive who wanted to bury the paper, etc.) and were stonewalled. To my knowledge these questions still haven't really been answered. I think its very reasonable request to want to know who or what caused your paper to get blocked. In normal peer review, even when it is blind, you can work with the reviewers to update the paper and see their feedback. These opportunities weren't readily provided.
  - sanity31415 5y ago
    So it's normal to demand the identities of a blind review panel? What exactly did I say that was "factually wrong" as you claim?
    - joshuamorton 5y ago
      > So it's normal to demand the identities of a blind review panel? It seems reasonable to ask "wait what blind review panel" when one appears out of thin air.
      - sanity31415 5y ago
        She did a lot more than that. Here is the email that got her fired: Hi friends, I had stopped writing here as you may know, after all the micro and macro aggressions and harassments I received after posting my stories here (and then of course it started being moderated). Recently however, I was contributing to a document that Katherine and Daphne were writing where they were dismayed by the fact that after all this talk, this org seems to have hired 14% or so women this year. Samy has hired 39% from what I understand but he has zero incentive to do this. What I want to say is stop writing your documents because it doesn’t make a difference. The DEI OKRs that we don’t know where they come from (and are never met anyways), the random discussions, the “we need more mentorship” rather than “we need to stop the toxic environments that hinder us from progressing” the constant fighting and education at your cost, they don’t matter. Because there is zero accountability. There is no incentive to hire 39% women: your life gets worse when you start advocating for underrepresented people, you start making the other leaders upset when they don’t want to give you good ratings during calibration. There is no way more documents or more conversations will achieve anything. We just had a Black research all hands with such an emotional show of exasperation. Do you know what happened since? Silencing in the most fundamental way possible. Have you ever heard of someone getting “feedback” on a paper through a privileged and confidential document to HR? Does that sound like a standard procedure to you or does it just happen to people like me who are constantly dehumanized? Imagine this: You’ve sent a paper for feedback to 30+ researchers, you’re awaiting feedback from PR & Policy who you gave a heads up before you even wrote the work saying “we’re thinking of doing this”, working on a revision plan figuring out how to address different feedback from people, haven’t heard from PR & Policy besides them asking you for updates (in 2 months). A week before you go out on vacation, you see a meeting pop up at 4:30pm PST on your calendar (this popped up at around 2pm). No one would tell you what the meeting was about in advance. Then in that meeting your manager’s manager tells you “it has been decided” that you need to retract this paper by next week, Nov. 27, the week when almost everyone would be out (and a date which has nothing to do with the conference process). You are not worth having any conversations about this, since you are not someone whose humanity (let alone expertise recognized by journalists, governments, scientists, civic organizations such as the electronic frontiers foundation etc) is acknowledged or valued in this company. Then, you ask for more information. What specific feedback exists? Who is it coming from? Why now? Why not before? Can you go back and forth with anyone? Can you understand what exactly is problematic and what can be changed? And you are told after a while, that your manager can read you a privileged and confidential document and you’re not supposed to even know who contributed to this document, who wrote this feedback, what process was followed or anything. You write a detailed document discussing whatever pieces of feedback you can find, asking for questions and clarifications, and it is completely ignored. And you’re met with, once again, an order to retract the paper with no engagement whatsoever. Then you try to engage in a conversation about how this is not acceptable and people start doing the opposite of any sort of self reflection—trying to find scapegoats to blame. Silencing marginalized voices like this is the opposite of the NAUWU principles which we discussed. And doing this in the context of “responsible AI” adds so much salt to the wounds. I understand that the only things that mean anything at Google are levels, I’ve seen how my expertise has been completely dismissed. But now there’s an additional layer saying any privileged person can decide that they don’t want your paper out with zero conversation. So you’re blocked from adding your voice to the research community—your work which you do on top of the other marginalization you face here. I’m always amazed at how people can continue to do thing after thing like this and then turn around and ask me for some sort of extra DEI work or input. This happened to me last year. I was in the middle of a potential lawsuit for which Kat Herller and I hired feminist lawyers who threatened to sue Google (which is when they backed off--before that Google lawyers were prepared to throw us under the bus and our leaders were following as instructed) and the next day I get some random “impact award.” Pure gaslighting. So if you would like to change things, I suggest focusing on leadership accountability and thinking through what types of pressures can also be applied from the outside. For instance, I believe that the Congressional Black Caucus is the entity that started forcing tech companies to report their diversity numbers. Writing more documents and saying things over and over again will tire you out but no one will listen. Timnit
Aunche 5y ago
>Being on the inside, we are often able to see downsides early before systems are widely deployed. Well they failed to catch that Microsoft Tae would be instantly turned into a internet Nazi bot.

rektide 5y ago

James Martin called it true in 2001 when he described the coming machine learned systems Alien Intelligences. unintelligible inhuman systems. from After the Internet: Alien Intelligence.

stephc_int13 5y ago

I agree with most of the content of this article. Especially this part: "Ethics are necessary, but not sufficient. More helpful are questions such as, who benefits and who is harmed by this AI system? And does it put power in the hands of the already powerful?" Distribution of power is the most important political question, more important than distribution of wealth or what we call ethics, that is always biased and hard to measure.

dec0dedab0de 5y ago
I agree, but wouldn't those questions, or atleast any opinions on the answers to those questions, just be part of ethics?
spoonjim 5y ago
But anything that benefits Microsoft, one of the most powerful companies in the world, is necessarily putting more power in the hands of the already powerful.

ThalesX 5y ago

> But from the beginning there was pushback and more recent work shows there is no reliable correlation between expressions on the face and what we are actually feeling. And yet we have tech companies saying emotions can be extracted simply by looking at video of people’s faces. We’re even seeing it built into car software systems. I've been involved with a startup where the CEO was certain we're only a step away from replacing humans in recruiting healthcare workers with AI analyzing expressions in self-submitted interview videos, as well as analysis of quizzes. I got pushed aside from that startup for being the 'obnixous technical humanist' (which is a caracterization I wear proudly). > AI is neither artificial nor intelligent. It is made from natural resources and it is people who are performing the tasks to make the systems appear autonomous. I've argued as a technological consultant times and times again that the effort they are spending on automating humans out, would be better spent in making those humans happier and augumenting them with an automated system. The fact that the CEO saw his low level employees as just temporary assets ready to be replaced meant that their life in the company was miserable. > This April, the EU produced the first draft omnibus regulations for AI I digress. I'm wondering what they're plan of action is since it was a European startup and since April we're theoretically pushed away from using AI to gauge candidates and all of their VC investment money came with the promise of 'automating healthcare recruitment and scaling globally'.

rgbrenner 5y ago

AI is neither artificial nor intelligent. It is made from natural resources and it is people who are performing the tasks to make the systems appear autonomous. Making something from natural resources does not make it natural. If that were the case, we wouldn't have the word "artificial", since everything we make comes from natural elements and everything would be "natural". The fact that you took natural resources and built something else from it--that it didn't exist in nature already--makes it artificial. AI is definitely artificial.

pkdpic_y9k 5y ago
That's a nice clean breakdown of that wording issue. I get her point though I guess, but I was hoping there might be something deeper.
deleted 5y ago
[deleted]
jcims 5y ago
I had the exact same reaction, but I wonder if I'm too close to it to appreciate what the term might conjure up in the broader (voting) public that isn't in the industry. (FWIW I kind of like the term synthetic intelligence as an alternative.) I do agree wholeheartedly that, much as in the case of cryptocurrency, there isn't an intuitive link between the product and the resources it requires to create and operate. The fact that GPT-3 would cost millions to reproduce in the commercial market doesn't even compute for me.
efnx 5y ago
I’m not arguing with your point, but let me offer another thought. The crux of this problem as I see it is that people consider themselves as separate from nature. The truth is that we are part of nature and the things we make are still part of nature. The opposite of “natural” is not “artificial” or “man-made” - it’s “supernatural”! Of course the word “artificial” is useful to classify things for our safety and benefit, but we are not supernatural and so the things we create still exist in nature - like we are the hand of the universe reconfiguring itself. This artificial separation of the human from nature has been popular in the past and I hope we overcome it.

smoldesu 5y ago

Honestly the hardest part about integrating AI into our daily lives is the fact that we don't really have AI yet. We have made great advances in the fields of machine learning and neural networks, but actually getting a computer to make educated decisions is hard. The current issue is that all of these models are black boxes: an ML model can guess which data will come out, but it can't really fully understand what it knows. It can identify slightly harder to notice patterns, but it can't actually think. We adopted the phrase "AI" far too early in the field, and I suspect there will be another few decades before we have the technical and scientific capability to make real artifical intelligence a thing.

otabdeveloper4 5y ago
> The current issue is that all of these models are black boxes Not quite. It'd be great if that was the only issue with neural network ML. A much bigger issue is that neural networks have extremely limited applicability. They're great for classification problems when you can have huge training datasets, but for many common problems they're useless. One obvious class of problems is time series prediction - extremely important in life and for business, and something neural networks are no good for.
- throwaway3699 5y ago
  Doesn't transfer learning level the playing field a bit? We already have excellent, off-the-shelf models for NLP, computer vision, ODT, etc... that just need fine-tuning for a particular business or domain problem. I think the 'huge training datasets' requirement is lessening every day. If you're doing something novel, then sure. I can see that.
  - otabdeveloper4 5y ago
    Predicting time series isn't "novel". It's the oldest data science problem in the world. And ML can't solve it yet.
    - 5y ago

jjcon 5y ago

> AI is neither artificial nor intelligent. It is made from natural resources and it is people who are performing the tasks to make the systems appear autonomous. People performing the tasks? As in coding tasks? That’s kinda the definition of artificial. If AI spawned naturally… that’s what wouldn’t be artificial. Or is she referring to data labeling and forgetting the many non supervised areas of AI? (Not that labeling would make it less artificial) Or is she suggesting that anything made from natural resources is natural… is anything not made from natural resources? The reason for artificial in artificial intelligence is to juxtapose against biological intelligence not speak to resource usage. I think arguing on the ‘not intelligent’ front is fine but that’s just kind of a word game. The field seeks intelligence and is fine making stupid ai if it means better ai later. Unless she is getting into notions of a soul or some bs about consciousness in which case we have left the realm of science altogether. Any way you slice it seems more like an inane point for making a headline than one for substantive discussion.

Cybotron5000 5y ago
It seems to me somewhat similar to an argument put forward by Jaron Lanier: that what is termed artificial intelligence’s (or ‘deep learning’’s etc.) current successes, at least as far as something like the translation services of the big corporations for example, is actually built by leveraging corpuses that are the fruits of a huge amount of individual human effort. Sometimes, say with the ‘Mechanical Turk’ thing that Amazon has, or CAPTCHAs or something, this is a more explicit connection (…bit Wizard of Oz! :) As I understand it, he proposes that an alternative to UBI etc. might be micro-compensations (or transactions) in return for providing this data. This might be a prelude/transition to a stage where our basic needs (on a sort of Mazlow’s hierarchy) were met by A.I., or that there might be increasingly creative or interesting ways to complete some tasks that we could go on refining forever.
- jjcon 5y ago
  > are the fruits of a huge amount of individual efforts That doesn’t make it any less artificial does it? If anything that makes it more so. That is also again only speaking to supervised learning (and only the fraction of it where datasets are curated not collected) not AI in general

mjburgess 5y ago

> Time and again, we see these systems producing errors ... and the response has been: “We just need more data.” but... you start to see forms of discrimination... in how they are built and trained to see the world. Thank goodness this perspective is getting out there. I have recently been incensed by the opposite view, that the bias is within "the data" only: https://www.youtube.com/watch?v=6jbin15-TcY https://www.youtube.com/watch?v=6jbin15-TcY . This is wholly false. Machines which analyse the world (ie., actual physical stuff, eg., people) in terms of statistical co-occurances within datasets cannot acquire the relevant understanding of the world. Consider NLP. It is likely that an NLP system analysing volumes of work on minority political causes will associate minority identifiers (eg., "black") with negative terms ("oppressed", "hostile", "against", "antagonistic"), etc. And thereby introduce an association which is not present within the text. This is because conceptual association is not statistical association. In such texts the conceptual association is "standing for", "opposing", "suffering from", "in need of". Not "likely to occur with". There are entire fields sold on a false equivocation between conceptual and statistical association. This equivocation generates novel unethical systems. AI systems are not mere symptoms of their data. They are unable, by design, to understand the data; and repeat it as-if it were a mere symptom of wordly co-occurance.

throwaway3699 5y ago
I don't know, inference from data is literally how all decisions are fundamentally made. Why wouldn't it be possible to create models that learn this particular pattern?
- mjburgess 5y ago
  Here's where the words "data", "pattern", etc. become unhelpful. Learning, as in what we do, is not learning associations in datasets. It is learning "associations" between: our body state and the world as we act. It's a sort of: (action, world, body, sensation, prior conceptualisation, ...) association. (Even then, our bodies grow and this is really not a formal process.) This is, at least, what is necessary to understand what words mean. Words are just tools that we use to coordinate with each other in a shared (physical) world. You really have to be here, with us, to understand them. Words mean what we do with them. Meaning has a "useful side-effect". It turns out when we are using words their sequencing reveals, on average, some commonalities in their use. Eg., when asking "Can you pass me the salt?" I may go on to ask, "and now the pepper". And thus there is a statistical association between the terms "salt" and "pepper". But a machine processing only those associations is completely unaware there is anything "salt" to pass, or even that there are objects in the world, or people, or anything. Really, the machine has no connection between its interior and the world, the very connection we have when we use words. When a machine generates the text "pass me the salt" it doesnt mean it. It cannot. There is no salt it's talking about. It doesnt even know what salt is. A machine used to make decisions concerning people, unaware of what a person even is, produces new unethical forms of action. Not merely just "being racist because the data is".

TOSSAWAY_1 5y ago

Lol, ai is racist Therefore the people making ai are racist. Smart. I'll weaponize accordingly.

ltbarcly3 5y ago

Ahh so it is naturally occurring without human artifice non-intelligence. Got it. NOWHANI

eunos 5y ago

It should not matter whether it is artificial or intelligence or neither, as long as it get the jobs done.

simonswords82 5y ago
Except when people are using the term AI to sell nonsense that doesn't do what it says on the tin.

version_five 5y ago

> ImageNet has now removed many of the obviously problematic people categories – certainly an improvement – however, the problem persists because these training sets still circulate on torrent sites [where files are shared between peers]. This is the scariest part of the article. The idea that some central authority should be censoring and revising datasets to keep up with political orthodoxy, and we should be rooting out unauthorized torrent sharing of unapproved training data. From a technical point of view, the common reason we pre-traini on imagenet is as the starting point for fine tuning for a specific use case. The diversity and size of the dataset makes good generic feature extractors. If you're using a ML model to identify people as kleptomaniac or drug dealer or other "problematic" labels, you're working on some kind of phrenology and it doesnt take an "AI ethicist" to know you shouldn't do it. But that's not the same as pretraining on imagenet, and certainly doesn't support trying to make datasets align with today's political orthodoxy.

Husafan 5y ago

Discuss amongst yourselves.

t_von_doom 5y ago

> The idea that you can see from somebody’s face what they are feeling is deeply flawed. I don’t think that’s possible. I agree with most of the article but this point I disagree with. Non verbal communication is a huge component in how we interact with each other. Covid example: When on calls where no cameras are on I get far less feedback when presenting to a 'room' compared to if I was to present in person or have cams on, even if the room stay silent in both situations. By looking at faces I can see who is distracted, who looks confused and whether what I am saying is being received well or poorly. Did that joke get smiles (polite or genuine ones?) or eye rolls? Now - do I think the current SOTA algorithms are at this level of nuance? No, definitely not. But to say it isn't at all possible is ridiculous in my opinion

cocoafleck 5y ago
I believe that people (usually) choose to communicate with faces, but just as people can lie with words they can lie with faces.

slver 5y ago

Even in situations where there's an objective reason for AI performing poorly with certain groups (like the worse light contrast on facial details with many photos of dark-skinned people), we go back to default and seek to cast moral judgment on someone for being racist, sexist, or something of the sort. Because you can't blame AI, we blame the people programming it, collecting data for it and training it. It seems we're trying to prevent a full-on moral panic that would be caused by the realization that "discrimination" is an objective need by the objective nature of the problem in some cases. For example there are about 14% black people in the US. If you have a FAIR SET OF DATA to train from, 14% of those faces would be black. No, they're not underrepresented, they're literally accurately represented. But this means the AI will be worse with those people. So what do we do? If we artificially up the "quota" and train with 50% black faces, the AI will now underperform with non-black faces. So then you need to train two networks: black faces and non-black faces, and then you need to "discriminate" between both, and pick the right network depending on the race you're working with. Math is racist sometimes.

throwaway3699 5y ago
Race aside, you can make this argument for any social strata. Gender, race, nationality, social class, etc... I don't believe most models will perform very different with a balanced training set.

yarg 5y ago

Yes it is, and of course it's not - it's artificial. If it were intelligent we'd be calling it "synthetic".

mark_l_watson 5y ago

I am reading her book and so far, I am enjoying it. The book really made me think of externalities for tech that I use and earn a living with. Costs of these externalities are environmental and human suffering. Recommended.

andyxor 5y ago

oh God, not again about "ethical AI", Gebru was fired for being a potz, who would want an employee throwing ultimatums and doxxing colleagues on social media