13 ms·

To me it seemed like Timnit Gebru was spoiling for any reason to fight, and decided to make a big deal out of a factually correct and hard-to-misinterpret state

by eric_b 6y ago

To me it seemed like Timnit Gebru was spoiling for any reason to fight, and decided to make a big deal out of a factually correct and hard-to-misinterpret statement. I mean, he was clearly referring to the specific model.

whymauri 6y ago
I'm in the group which was upset over the insinuation that ML Researchers should not care about fairness or ethics as much as ML Engineers. The distinction between researcher and engineer should not be grounds to care or not care about safe and ethical model building. This is especially true when the difference between either role is highly arbitrary and varies by organization or field. For the sake of ethical R&D, it's counter-productive to build a hierarchy of investment into the problem. Admittedly, the responsibility of end-results can differ, but the consensus that this ethical work is important should ideally be universal. That said, there should not be some ivory tower where you wash your hands of the ethical problems of your field.
- eric_b 6y ago
  I don't understand what Timnit and these other people are really even advocating. She says "diverse datasets are not enough". Structural problems can't be ignored etc. Ok, so what is the solution then? Does she have a concrete set of steps or goals to address the problems she sees? Is there a list somewhere of things that would appease her, and in her mind make ML fair? Honestly interested.
  - whymauri 6y ago
    I can't speak for Timnit, but for me it would have sufficed to see Yann start with "I care about ethics and so should researchers and engineers." Instead it took so long to get there, so many "buts" and so many implications that researchers shouldn't be as invested, that I was disappointed. As for Timnit and LeCun: a user above notes that the epistemological frameworks they're using to analyze this problem are not aligned. I found that comment pretty eye-opening, honestly.
    - free_rms 6y ago
      Why does everything need 50 disclaimers before anyone can say anything. Shouldn't "I care about ethics" just be assumed? How many people do you know who would say "I don't care about ethics"?
      - whymauri 6y ago
        My experience in ML and computer science is that it cannot be assumed. Consider companies like Palantir and their Gotham system. Consider the facial recognition systems deployed in Detroit, which fail 96% of thee time. It especially cannot be assumed when you attempt to offload that responsibility to other people. Computer science programs around the country promise their young, bright-eyed undergrads that they'll change the world. Very few of them teach them the ethics they'll need to do that in a thoughtful way. The assumption that science and technology is inherently ethical has unfortunately led to dangerous ideas over the past 150 years. Some, like geographic determinism and eugenics, have directly led to the suffering of millions of people. I hope we tread carefully and take action when we see harmful models. [0] [0] https://twitter.com/SpringerNature/status/1275477365196566528 https://twitter.com/SpringerNature/status/127547736519656652...
        free_rms 6y ago
        If you're saying CS people need to read more humanities, I'm absolutely with you. But I didn't say that science and technology are inherently ethical. I said that most people care about ethics. They might have different priorities or philosophy than you, but almost nobody commenting on a social issue is doing so because they want the immoral thing to happen. Right? So asking everyone to say "of course I care, of course" before everything they say is laborious.
        guerrilla 6y ago
        I don't think immoral is the problem but rather amoral. In this case, that ethics was an afterthought.
  - stonogo 6y ago
    The solution they seek is to stop pretending that research happens in a vacuum, to think about likely applications of the technologies we develop, and stop taking for granted the idea that the invisible hand of the market will determine the optimum application of the cool shit we make. This is an extremely unpleasant position to take, if your point of view is empowered within the status quo. It is much extra work for no discernible benefit to the researcher. If your point of view is subject to disproportionate suffering under the status quo, then reinforcing current practices by implicitly enshrining them in input datasets will make improving your situation even harder. As an example, consider the case of public school funding. In a hypothetical system where school resources are provided proportionally based on student success, good schools will thrive and bad schools will get worse. If someone points out this isn't fixing the problem, you can reverse the proportions -- this will cause good schools to suffer while bad schools will get additional funding (disincentivizing student success). In cases like this, it's not enough to just have a purely abstract set of metrics on which to base resource allocation: it will always require actual investigation of why good schools produce good results and why students do poorly in specific schools. This is sort of obvious, of course, but it isn't being translated into terms that some researchers can or will grasp. It's never enough to just tell someone to 'debias the dataset,' as determining that bias is a monumentally difficult challenge that people have failed to achieve for many generations. A key factor in fact is the propensity for this kind of research to get deployed, today, by people who are not experts in a given domain of investigation, with possibly disastrous results in policymaking. These tools are not abstractions that require a team of experts to translate from research paper to the real world; ML researchers put out results that you can shove into your nearest computer and run. What Timnit and others are getting at is that it requires thoughtful and careful assessment to get real value out of this sort of research. Ideally, in Timnit's assessment, the researchers themselves would put effort into identifying possible calamities and put as much effort into mitigating them as they do into publicizing the work itself. Yann LeCun and other researchers simply do not believe this is their responsibility; all they want to focus on is the mathematics themselves. I'm sympathetic to this position but I also very much do understand the opposition. One of my favorite movies from childhood, "Real Genius," deals with this sort of issue as the main plot line.
    - deleted 6y ago
  - toofy 6y ago
    Lacking an immediate solution to a complex problem does not mean these problems don’t exist. I’ve noticed often lately that people, when talking past each other, one person is saying, “We really need to consider the implications of problem X.” and the other person will imply “If there is no obvious solution, then no one should consider this a problem.” When it comes to problems, particularly in complex subjects which aren’t yet well understood, and where there aren’t yet an overabundance of high caliber researchers, this comes across as dismissive of the problem. This can be even more concerning if we know there are investors lined up who will happily sell something to the world and who will intentionally hide or minimize known ethical concerns. And then play dumb and shocked later when the very same problems manifest. I see this dismissal of justifiable and real concerns an awful lot in conversations of all kinds lately. And from what I’ve seen, no one here is anti-ML, no one on either side here is a luddite. But like so many conversations online, we should quit talking past each other and likely need to quit trying to paint people as if their concerns don’t have very real ethical implications which will, if left unaddressed, manifest in all kinds of negative ways throughout society. Again, a lack of a neat and tidy solution doesn’t mean the problem doesn’t exist.
  - AlexCoventry 6y ago
    They go into more detail in this talk: https://www.youtube.com/watch?v=vpPpwa7W93I https://www.youtube.com/watch?v=vpPpwa7W93I Specific recommendations start at 17 min.
- unishark 6y ago
  > This is especially true when the difference between either role is highly arbitrary and varies by organization or field. I don't think this criticism is fair. Presumably if someone with the title "researcher" has a hand in actually doing what LeCun consider to be the engineer's role, LeCun would say to treat them as an engineer for the purposes of his argument.
  - whymauri 6y ago
    The distinction is fundamentally pointless because the lines are blurred and highly subjective. I'll repeat it again: >For the sake of ethical R&D, it's counter-productive to build a hierarchy of investment into the problem. Admittedly, the responsibility for end-results can differ, but the consensus that this ethical work is important should ideally be universal. That said, there should not be some ivory tower where you wash your hands of the ethical problems of your field. Suppose you're correct, should we define this threshold based on just LeCun's heuristic for defining it? Or would it likely be better to have a consensus that your role doesn't matter in acknowledging the important of fairness and ethics? Would you prefer the world's most prolific researchers being mindful of these issues, even subconsciously? Or to care less, perhaps very little, because it can be deferred to engineers? Because I would like for the field to unite against building harmful systems and to acknowledge the importance of this work throughout the academic hierarchy.
    - unishark 6y ago
      The concern in your quote is separate from the question of whether we can differentiate between engineers and researches. As for the ethical problems. Are you familiar with machine learning methods? It really is all about the data; that's not a dismissal, it's a fact about current technology. There are other kinds of A.I. which are not based entirely on raw data like this. Machine Learning has been simplistically described as "curve fitting", which I think isn't a bad description. So here you are arguing that the mathematician researching good ways to fit a smooth curve to a series of points in really high dimensions needs to somehow take into account what those points might represent in someone's use of the technology. It seems pretty unreasonable to me to require that they put ethical constraints on it.
cgearhart 6y ago
The core distinction is understanding that bias in ML is not _just_ bias in the data. It may be true that we can reduce bias in this model by reducing bias in the training data, but there is a deeper, more fundamental problem of bias that will not be solved just by changing the training data. Marginalizing the discussion by pointing out that this case would benefit from less biased data is unproductive.
- eric_b 6y ago
  I asked another commenter as well, but what are the proposed solutions then? People are obviously upset about ML and bias, is there a place I can get a summary of actionable next steps to lessen bias in ML?
  - cgearhart 6y ago
    There’s not one neat trick to make it go away. There have been a number of fairness and bias workshops and forums in recent ML conferences. There’s also a growing podcast and book collection on the topic. Timnit Gebru (mentioned in the OP article) has published and participated in a bunch, maybe start there.
    - Nasrudith 6y ago
      To be frank phrasing it like that makes it sound like Gebru has an ulterior motive in trying to cash in books/speakers fees/consulting. Even if sincere good faith and he has real expertise is assumed that approach kind of raises several "huckster alert" red flags.
- AlexCoventry 6y ago
  What are some examples of racial bias in ML models which cannot be solved by just changing the training data?
- unishark 6y ago
  > Marginalizing the discussion by pointing out that this case would benefit from less biased data is unproductive. Except for the part where it provides an actual workable solution to the problem at hand. To me this is an argument between completely different mindsets, one that restricts itself to provable facts and one which restricts itself to political agendas. I don't see how the latter can also work in facts. Or belongs in a technical research discussion at all frankly. You want to make laws that force companies to produce identical/equivalent outcomes for every race somehow? Just go lobby for it. Perhaps it's a good idea. You aren't going to reprogram mathematicians to think in political terms instead of mathematical terms.
curiousgal 6y ago
Just browsing her Twitter made me cringe and I am a female PoC. Her response to every argument is "Ah yes the problem with the world is a PoC doing this and that" when the original argument hasn't even suggested that. Recently seems in support of a black scholar who cried racism because her non peer-reviewed work that was only posted on arxiv wasn't cited in a lecture on GANs ... Voicing these opinions would probably label me as racist in their book ironically.
- dmix 6y ago
  > Voicing these opinions would probably label me as racist in their book ironically. Don't worry, Twitter isn't real life. People have become experts at shouting down people who disagree with them on that platform, they aren't seeking proper arguments and make disingenuous attempts to present them as honest debates. But fortunately what's popular on Twitter doesn't translate to the average population. Plenty of completely fringe ideas get 50-100k likes/retweets. It mostly just represents the voices of various super-niches living in bubbles.
  - blululu 6y ago
    >>Don't worry, Twitter isn't real life. For the most part yes, but President Trump's behavior on Twitter can have serious consequences in the real world. I think that the behavioral norms of social media are penetrating deeper and deeper into culture.
    - dmix 6y ago
      You seem to be taking "not real life" too literally. I'm not saying "nothing on Twitter affects real life". The point is that it's often a poor representation of real life. Small but highly vocal groups can have a seemingly loud and powerful voice. Yet the results of polls and other public signals (even election outcomes) are frequent reminders that what is gospel on Twitter is often detached from 'real life'. Using a president of a major country is a poor example in this context. But if anything Trump being one of the first major Twitter users strongly reinforces my point. Prior to election he tweeted plenty of things most mainstream US republicans wouldn't touch with a 10 foot poll. Let alone what an average American would say IRL (even right leaning ones). Not to mention Twitter is a global platform so conversation around local politics can be heavily skewed by people not even in the country. But otherwise I agree, it is infesting real life far more frequently these days. And it is worrying. Despite everything I said above, big corporations, the media, politicians, etc can't seem to make this distinction and take what is popular there as a direct reflection of the general public. And it creates a negative reinforcing spiral.
ameen 6y ago
As a POC, it is exhausting to have folks like Timnit spin everything into a problem devoid of a solution. I hope we take a scientific approach to problems and solve it rather than driving away folks who can help in solving the problem.
- thegayngler 6y ago
  As a POC, it bothers me that someone from the "corporate(or educated) class" of POCs always coming in to criticize those POC who have a legitimate concern about some injustice they are seeing. In this case Timmits concern is entirely valid and does not have a solution currently other than do not use ML for some applications. We benefit from Timmit and others voicing these valid concerns. Engineers always want to base everything on the data to take themselves out of what is being asked of them. It is not always possible to reduce problems to "the data". POC in particular should not be quiet when it comes to some of the issues around the questionable use of ML in relation to race and issues that are ultimately surrounding race.
- newen 6y ago
  It makes sense once you realize she is only using this to get an executive position in some institute for removing racial biases in AI and be set for life.

by eric_b 6y ago

whymauri 6y ago

I'm in the group which was upset over the insinuation that ML Researchers should not care about fairness or ethics as much as ML Engineers. The distinction between researcher and engineer should not be grounds to care or not care about safe and ethical model building. This is especially true when the difference between either role is highly arbitrary and varies by organization or field. For the sake of ethical R&D, it's counter-productive to build a hierarchy of investment into the problem. Admittedly, the responsibility of end-results can differ, but the consensus that this ethical work is important should ideally be universal. That said, there should not be some ivory tower where you wash your hands of the ethical problems of your field.

eric_b 6y ago
I don't understand what Timnit and these other people are really even advocating. She says "diverse datasets are not enough". Structural problems can't be ignored etc. Ok, so what is the solution then? Does she have a concrete set of steps or goals to address the problems she sees? Is there a list somewhere of things that would appease her, and in her mind make ML fair? Honestly interested.
- whymauri 6y ago
  I can't speak for Timnit, but for me it would have sufficed to see Yann start with "I care about ethics and so should researchers and engineers." Instead it took so long to get there, so many "buts" and so many implications that researchers shouldn't be as invested, that I was disappointed. As for Timnit and LeCun: a user above notes that the epistemological frameworks they're using to analyze this problem are not aligned. I found that comment pretty eye-opening, honestly.
  - free_rms 6y ago
    Why does everything need 50 disclaimers before anyone can say anything. Shouldn't "I care about ethics" just be assumed? How many people do you know who would say "I don't care about ethics"?
    - whymauri 6y ago
      My experience in ML and computer science is that it cannot be assumed. Consider companies like Palantir and their Gotham system. Consider the facial recognition systems deployed in Detroit, which fail 96% of thee time. It especially cannot be assumed when you attempt to offload that responsibility to other people. Computer science programs around the country promise their young, bright-eyed undergrads that they'll change the world. Very few of them teach them the ethics they'll need to do that in a thoughtful way. The assumption that science and technology is inherently ethical has unfortunately led to dangerous ideas over the past 150 years. Some, like geographic determinism and eugenics, have directly led to the suffering of millions of people. I hope we tread carefully and take action when we see harmful models. [0] [0] https://twitter.com/SpringerNature/status/1275477365196566528 https://twitter.com/SpringerNature/status/127547736519656652...
      - free_rms 6y ago
        If you're saying CS people need to read more humanities, I'm absolutely with you. But I didn't say that science and technology are inherently ethical. I said that most people care about ethics. They might have different priorities or philosophy than you, but almost nobody commenting on a social issue is doing so because they want the immoral thing to happen. Right? So asking everyone to say "of course I care, of course" before everything they say is laborious.
        guerrilla 6y ago
        I don't think immoral is the problem but rather amoral. In this case, that ethics was an afterthought.
- stonogo 6y ago
  The solution they seek is to stop pretending that research happens in a vacuum, to think about likely applications of the technologies we develop, and stop taking for granted the idea that the invisible hand of the market will determine the optimum application of the cool shit we make. This is an extremely unpleasant position to take, if your point of view is empowered within the status quo. It is much extra work for no discernible benefit to the researcher. If your point of view is subject to disproportionate suffering under the status quo, then reinforcing current practices by implicitly enshrining them in input datasets will make improving your situation even harder. As an example, consider the case of public school funding. In a hypothetical system where school resources are provided proportionally based on student success, good schools will thrive and bad schools will get worse. If someone points out this isn't fixing the problem, you can reverse the proportions -- this will cause good schools to suffer while bad schools will get additional funding (disincentivizing student success). In cases like this, it's not enough to just have a purely abstract set of metrics on which to base resource allocation: it will always require actual investigation of why good schools produce good results and why students do poorly in specific schools. This is sort of obvious, of course, but it isn't being translated into terms that some researchers can or will grasp. It's never enough to just tell someone to 'debias the dataset,' as determining that bias is a monumentally difficult challenge that people have failed to achieve for many generations. A key factor in fact is the propensity for this kind of research to get deployed, today, by people who are not experts in a given domain of investigation, with possibly disastrous results in policymaking. These tools are not abstractions that require a team of experts to translate from research paper to the real world; ML researchers put out results that you can shove into your nearest computer and run. What Timnit and others are getting at is that it requires thoughtful and careful assessment to get real value out of this sort of research. Ideally, in Timnit's assessment, the researchers themselves would put effort into identifying possible calamities and put as much effort into mitigating them as they do into publicizing the work itself. Yann LeCun and other researchers simply do not believe this is their responsibility; all they want to focus on is the mathematics themselves. I'm sympathetic to this position but I also very much do understand the opposition. One of my favorite movies from childhood, "Real Genius," deals with this sort of issue as the main plot line.
  - deleted 6y ago
- toofy 6y ago
  Lacking an immediate solution to a complex problem does not mean these problems don’t exist. I’ve noticed often lately that people, when talking past each other, one person is saying, “We really need to consider the implications of problem X.” and the other person will imply “If there is no obvious solution, then no one should consider this a problem.” When it comes to problems, particularly in complex subjects which aren’t yet well understood, and where there aren’t yet an overabundance of high caliber researchers, this comes across as dismissive of the problem. This can be even more concerning if we know there are investors lined up who will happily sell something to the world and who will intentionally hide or minimize known ethical concerns. And then play dumb and shocked later when the very same problems manifest. I see this dismissal of justifiable and real concerns an awful lot in conversations of all kinds lately. And from what I’ve seen, no one here is anti-ML, no one on either side here is a luddite. But like so many conversations online, we should quit talking past each other and likely need to quit trying to paint people as if their concerns don’t have very real ethical implications which will, if left unaddressed, manifest in all kinds of negative ways throughout society. Again, a lack of a neat and tidy solution doesn’t mean the problem doesn’t exist.
- AlexCoventry 6y ago
  They go into more detail in this talk: https://www.youtube.com/watch?v=vpPpwa7W93I https://www.youtube.com/watch?v=vpPpwa7W93I Specific recommendations start at 17 min.
unishark 6y ago
> This is especially true when the difference between either role is highly arbitrary and varies by organization or field. I don't think this criticism is fair. Presumably if someone with the title "researcher" has a hand in actually doing what LeCun consider to be the engineer's role, LeCun would say to treat them as an engineer for the purposes of his argument.
- whymauri 6y ago
  The distinction is fundamentally pointless because the lines are blurred and highly subjective. I'll repeat it again: >For the sake of ethical R&D, it's counter-productive to build a hierarchy of investment into the problem. Admittedly, the responsibility for end-results can differ, but the consensus that this ethical work is important should ideally be universal. That said, there should not be some ivory tower where you wash your hands of the ethical problems of your field. Suppose you're correct, should we define this threshold based on just LeCun's heuristic for defining it? Or would it likely be better to have a consensus that your role doesn't matter in acknowledging the important of fairness and ethics? Would you prefer the world's most prolific researchers being mindful of these issues, even subconsciously? Or to care less, perhaps very little, because it can be deferred to engineers? Because I would like for the field to unite against building harmful systems and to acknowledge the importance of this work throughout the academic hierarchy.
  - unishark 6y ago
    The concern in your quote is separate from the question of whether we can differentiate between engineers and researches. As for the ethical problems. Are you familiar with machine learning methods? It really is all about the data; that's not a dismissal, it's a fact about current technology. There are other kinds of A.I. which are not based entirely on raw data like this. Machine Learning has been simplistically described as "curve fitting", which I think isn't a bad description. So here you are arguing that the mathematician researching good ways to fit a smooth curve to a series of points in really high dimensions needs to somehow take into account what those points might represent in someone's use of the technology. It seems pretty unreasonable to me to require that they put ethical constraints on it.

cgearhart 6y ago

The core distinction is understanding that bias in ML is not _just_ bias in the data. It may be true that we can reduce bias in this model by reducing bias in the training data, but there is a deeper, more fundamental problem of bias that will not be solved just by changing the training data. Marginalizing the discussion by pointing out that this case would benefit from less biased data is unproductive.

eric_b 6y ago
I asked another commenter as well, but what are the proposed solutions then? People are obviously upset about ML and bias, is there a place I can get a summary of actionable next steps to lessen bias in ML?
- cgearhart 6y ago
  There’s not one neat trick to make it go away. There have been a number of fairness and bias workshops and forums in recent ML conferences. There’s also a growing podcast and book collection on the topic. Timnit Gebru (mentioned in the OP article) has published and participated in a bunch, maybe start there.
  - Nasrudith 6y ago
    To be frank phrasing it like that makes it sound like Gebru has an ulterior motive in trying to cash in books/speakers fees/consulting. Even if sincere good faith and he has real expertise is assumed that approach kind of raises several "huckster alert" red flags.
AlexCoventry 6y ago
What are some examples of racial bias in ML models which cannot be solved by just changing the training data?
unishark 6y ago
> Marginalizing the discussion by pointing out that this case would benefit from less biased data is unproductive. Except for the part where it provides an actual workable solution to the problem at hand. To me this is an argument between completely different mindsets, one that restricts itself to provable facts and one which restricts itself to political agendas. I don't see how the latter can also work in facts. Or belongs in a technical research discussion at all frankly. You want to make laws that force companies to produce identical/equivalent outcomes for every race somehow? Just go lobby for it. Perhaps it's a good idea. You aren't going to reprogram mathematicians to think in political terms instead of mathematical terms.

curiousgal 6y ago

Just browsing her Twitter made me cringe and I am a female PoC. Her response to every argument is "Ah yes the problem with the world is a PoC doing this and that" when the original argument hasn't even suggested that. Recently seems in support of a black scholar who cried racism because her non peer-reviewed work that was only posted on arxiv wasn't cited in a lecture on GANs ... Voicing these opinions would probably label me as racist in their book ironically.

dmix 6y ago
> Voicing these opinions would probably label me as racist in their book ironically. Don't worry, Twitter isn't real life. People have become experts at shouting down people who disagree with them on that platform, they aren't seeking proper arguments and make disingenuous attempts to present them as honest debates. But fortunately what's popular on Twitter doesn't translate to the average population. Plenty of completely fringe ideas get 50-100k likes/retweets. It mostly just represents the voices of various super-niches living in bubbles.
- blululu 6y ago
  >>Don't worry, Twitter isn't real life. For the most part yes, but President Trump's behavior on Twitter can have serious consequences in the real world. I think that the behavioral norms of social media are penetrating deeper and deeper into culture.
  - dmix 6y ago
    You seem to be taking "not real life" too literally. I'm not saying "nothing on Twitter affects real life". The point is that it's often a poor representation of real life. Small but highly vocal groups can have a seemingly loud and powerful voice. Yet the results of polls and other public signals (even election outcomes) are frequent reminders that what is gospel on Twitter is often detached from 'real life'. Using a president of a major country is a poor example in this context. But if anything Trump being one of the first major Twitter users strongly reinforces my point. Prior to election he tweeted plenty of things most mainstream US republicans wouldn't touch with a 10 foot poll. Let alone what an average American would say IRL (even right leaning ones). Not to mention Twitter is a global platform so conversation around local politics can be heavily skewed by people not even in the country. But otherwise I agree, it is infesting real life far more frequently these days. And it is worrying. Despite everything I said above, big corporations, the media, politicians, etc can't seem to make this distinction and take what is popular there as a direct reflection of the general public. And it creates a negative reinforcing spiral.

ameen 6y ago

As a POC, it is exhausting to have folks like Timnit spin everything into a problem devoid of a solution. I hope we take a scientific approach to problems and solve it rather than driving away folks who can help in solving the problem.

thegayngler 6y ago
As a POC, it bothers me that someone from the "corporate(or educated) class" of POCs always coming in to criticize those POC who have a legitimate concern about some injustice they are seeing. In this case Timmits concern is entirely valid and does not have a solution currently other than do not use ML for some applications. We benefit from Timmit and others voicing these valid concerns. Engineers always want to base everything on the data to take themselves out of what is being asked of them. It is not always possible to reduce problems to "the data". POC in particular should not be quiet when it comes to some of the issues around the questionable use of ML in relation to race and issues that are ultimately surrounding race.
newen 6y ago
It makes sense once you realize she is only using this to get an executive position in some institute for removing racial biases in AI and be set for life.