16 ms·

The machine learning community has a toxicity problem

143 points by zekrioca 6y ago

baylearn 6y ago
Agree with this point: Sixthly, moral and ethics are set arbitrarily. The U.S. domestic politics dominate every discussion. At this very moment, thousands of Uyghurs are put into concentration camps based on computer vision algorithms invented by this community, and nobody seems even remotely to care. Adding a "broader impact" section at the end of every people will not make this stop. There are huge shitstorms because a researcher wasn't mentioned in an article. Meanwhile, the 1-billion+ people continent of Africa is virtually excluded from any meaningful ML discussion (besides a few Indaba workshops).
- LatteLazy 6y ago
  Maybe I am misunderstanding but... Why does a community need a moral stance on external issues? If everyone is there to do/learn/advance machine learning, isn't that the end of it? It isn't their purpose to engage with Chinese or other issues. Edit: I think it makes sense to have a stance on Machine Learning moral issues in a machine learning group. I just don't think you need to have a stance on every issue ever. I say this as someone always a bit frustrated that HN and Reddit etc are very US centric.
  - fennecfoxen 6y ago
    If a community has a stance on US politics which interact with it only incidentally — which the community very much does have, to the point of distraction, it is alleged — then the community can damn well afford to have a stance on international politics when the advancements the community builds are key tools in building the engines of tyrannical repression.
    - LatteLazy 6y ago
      I think it's fine to take a stance on things central to the groups main function: don't supply ML Tech for immoral use. But beyond that? That's a lot of issues you need to make stances on before you can actually do any machine learning work...
      - SpicyLemonZest 6y ago
        This issue isn't beyond that. The question is how a commitment to not supply ML tech for immoral use is compatible with supplying it to a country that's genociding the Uighurs.
      - crispyambulance 6y ago
        > don't supply ML Tech for immoral use. But beyond that? There's a lot to that and it goes beyond the scope of what researchers can handle by themselves. It's an effort that involves more than just the researchers, though they will take a central role. For one thing, how do you evaluate whether a usage is "immoral"? How do you enforce correct usage? These are difficult questions with multifaceted answers which no one has yet elucidated (yes, it has to be more than "I'll know it when I see it").
        LatteLazy 6y ago
        This is exactly my point. You can get general agreement to "do no evil". But every step you take to be specific makes the amount of discussion and detail exponentially more difficult. So you can't have a specific stance on everything. You either take a specific stance on a small number of issues central to the community or a general stance, or you spend all your time creating and amending stances and do nothing else.
  - andyjohnson0 6y ago
    > Maybe I am misunderstanding but... Why does a community need a moral stance on external issues? If everyone is there to do/learn/advance machine learning, isn't that the end of it? It isn't their purpose to engage with Chinese or other issues. Because its wrong to write-off the mistreatment of other human beings as an external issue. It never is: we're all human and we all share the same planet.
    - LatteLazy 6y ago
      So any group anywhere needs to have a stance on every injustice everywhere? Isn't that exhausting?
      - SiempreViernes 6y ago
        You just need to take a stance on injustice classes, after that its simply classification. Surely you can muster that much moral effort?
        LatteLazy 6y ago
        Sorry, what's a class in this context I'm more than happy to say something like "injustice everywhere is bad and we should oppose it and not let our technology be used to support it". But I feel that's a bit less than people expect?
      - crispyambulance 6y ago
        > Isn't that exhausting? Eventually, but this is a unique time right now. People are paying attention to injustice. I would say that much of these "stances" are inauthentic and done just for PR reasons. We'll see who is being genuine and who is phony soon enough.
        protomyth 6y ago
        No, people are paying attention to some injustice. They don't give a damn about other injustices unless they can wrap those back into one of their causes. They'll visit Native Americans when its something about environment, but ignore them when it comes to police violence.
  - deleuze 6y ago
    How the technology you develop is used is an "external issue"? Ivory tower much?
    - kiliantics 6y ago
      Would you say the issues are more "rhizomatic"? ;)
  - crispyambulance 6y ago
    > Why does a community need a moral stance on external issues? Some communities choose to do so. Einstein, Szilard, Pauling (and others) formed the Emergency Committee of Atomic Scientists to argue against nuclear weapons at the dawn of the Atomic era. And many scientists have taken a stand against nuclear weapons since then. Their actions have, at least, influenced the proliferation of these weapons and informed the public about their dangers. Would you argue that it wasn't "their business?" One could make a case that machine learning has negative consequences/possibilities that are similar in scope to nuclear weapons. The people who work with and understand machine learning are in the best position to take a stand against abuse of these technologies. I would say it's their responsibility to speak out, otherwise a despot will do it for them.
    - 0-_-0 6y ago
      > Their actions have, at least, influenced the proliferation of these weapons How? The number of nuclear weapons was growing rapidly until the 80s, to the point where dozens of bombs were aimed at single targets. > informed the public about their dangers Seeing the effects of bombs informed the public, not a committee
      - crispyambulance 6y ago
        > How? The number of nuclear weapons was growing rapidly until the 80s. It's hard to assess their impact, but that doesn't mean it was insignificant. Could we have been in a worse position? Definitely. Someone might have "pushed the button" by now were it not for decades of work from organizations (some of whom consisted of scientists) lobbying against nuclear weapons and their proliferation. > Seeing the effects of bombs informed the public, not a committee The nuclear tests demonstrated that the weapons work. There are other effects such as "nuclear winter" that have not been (and hopefully never will be) demonstrated. Sadly, because of Chernobyl, we do have examples of what happens to contaminated areas. Understanding and communicating the effects beyond "blast radius" is best done by scientists and health experts.
  - SpicyLemonZest 6y ago
    Not every community does. The ML community generally believes they're obligated to have moral stances on external issues because they believe machine learning has great potential to cause harm if not properly controlled.
    - LatteLazy 6y ago
      Thanks. I guess I can admire the intent and the recognition of consequences of important tech. Honestly though, I'm glad I'm not responsible for organising that sort of response...
  - whytaka 6y ago
    The world is changing vigorously. Trust is breaking down domestically and internationally. For some, it was never trust but an oppressive stability. People are anxious about who they are dealing with because who rises to power now will determine which subcultures will enjoy great political prominence and who will have their voices silenced in the near future. The battle is over values, standards, and world views. A lot is at stake. Technology is the ultimate medium now. Technologies like machine learning shapes discussions like money and GDP figures continues to shape discussions about the state of societies the world over. Ignore the strategically important position that technology holds now and in the future of our society at your own peril. To be so willfully blind to the changing times, I can’t imagine a less engaged worldview.
  - michaelmrose 6y ago
    If your research is used to create such ill that on net throwing every researcher in your field into the sea would be a lesser evil the question of external issues at least deserves a look lest you miss an opportunity to avert harm. Note I do not support tossing ml researchers into the sea and lots of industries are worse.
- Udik 6y ago
  > thousands of Uyghurs are put into concentration camps based on computer vision algorithms invented by this community I saw this repeated frequently in the last few days, and I think I've missed some important info. I know that China uses face recognition for surveillance purpose, and that it detains a substantial share of the Uighur population for supposed violations of the law. I miss a link between the two: is China using AI to detect who is Uighur and arrest him/ her for this very fact? Does anyone have more info?
  - NikolaeVarius 6y ago
    https://www.nytimes.com/2019/04/14/technology/china-surveillance-artificial-intelligence-racial-profiling.html https://www.nytimes.com/2019/04/14/technology/china-surveill...
    - Udik 6y ago
      Thank you, but this article seems to say that the technology is used to track Uighurs (I suppose for the reason that some of them have been engaging in terrorist activities) and not to incarcerate them. Despite the scale- which we considered worrying in itself and for understandable reasons- this seems to be an automated version of "I suddenly see a lot of mafioso-looking guys congregating here, let's keep an eye to see if there's anything dodgy going on"- which has been practiced with skilled eyes since forever. And it's a whole different thing from "if I see anyone looking Jewish I'll round them up and send them to a concentration camp".
      - cycomanic 6y ago
        > Thank you, but this article seems to say that the technology is used to track Uighurs (I suppose for the reason that some of them have been engaging in terrorist activities) and not to incarcerate them. That's a weird suggestion. It's undisputed that they have put millions into detention camps, so are you suggesting they all have been engaging in "terrorist activities"? So why suggest ("suppose" ) that they have been engaging in terrorist activities? > Despite the scale- which we considered worrying in itself and for understandable reasons- this seems to be an automated version of "I suddenly see a lot of mafioso-looking guys congregating here, let's keep an eye to see if there's anything dodgy going on"- which has been practiced with skilled eyes since forever. And it's a whole different thing from "if I see anyone looking Jewish I'll round them up and send them to a concentration camp". The article literally said they are looking for (tracking) Uighurs because of their ethnic appearance. That is looking for "Jewish" , not looking "mafioso-looking" in your example above
craftinator 6y ago
I would argue that anything that becomes sufficiently popular enough to be lucrative beyond some arbitrary threshold becomes toxic. Welcome to the cycle of capitalism!
- godelzilla 6y ago
  Progress < profits and control
  - DudeInBasement 6y ago
    Just a series of weighted if statements anyway...
tomp 6y ago
> At our CS faculty, only 30% of undergrads and 15% of the professors are women. Going on parental leave during a PhD or post-doc usually means the end of an academic career. However, this lack of diversity is often abused as an excuse to shield certain people from any form of criticism. Reducing every negative comment in a scientific discussion to race and gender creates a toxic environment. People are becoming afraid to engage in fear of being called a racist or sexist, which in turn reinforces the diversity problem. This was an unexpected twist! It's rare to read such an honest, unbiased opinion on this issue.
- deleuze 6y ago
  I'm glad you've honed in to identify that the 15-30% of women in machine learning are actually the real problem. Which seems consistent with your previous claim that: > I'm a man, but I've never received any benefit just because (most of) our presidents / businessmen are men (in fact, arguably (individual) women receive more benefits, because most of those men are heterosexuals). A small minority of women are actually the real privileged ones, huh?
  - new2628 6y ago
    They are not the real problem. The all-or-nothing exaggerations that remove all nuance are (a major part of) the real problem.
    - deleuze 6y ago
      I think the person saying that "akshually, women are more privileged because I'm a heterosexual" is the one lacking nuance.
      - SpicyLemonZest 6y ago
        Nobody said this; you're inventing strawmen. e: Apparently someone did say this. Apologies.
        6y ago
mellosouls 6y ago
This is very refreshing to read - looking from the outside in, there has been a pretty clear problem with (at least the representation of) ML research along the lines highlighted here - ridiculous hype, papers again and again linked from the same researchers and institutions indicating potential cliqueyness, nepotism and celebrity worship. All very dispiriting, indicating real problems making progress in AGI beyond the clear lack of ideas. Of course, just because these toxicity claims are being made, it doesn't mean they are accurate, but they certainly ring true. If they are, it is good to know they are being talked about in some quarters.
- foolmeonce 6y ago
  > Of course, just because these toxicity claims are being made, it doesn't mean they are accurate, but they certainly ring true. In many respects I think it is more important that we have no idea what is true about any industry. When I dwell too much about my impression of any part of the tech field I usually end up demotivated to do anything that might connect me with the social group involved. I think there is a general shallowness and overcompensation in our industry that makes it impossible to figure out what real life is like with a group until you are way too invested and they have already hazed you.
- zippy5 6y ago
  I'm super curious why you were optimistic about AGI in the first place? it seem to me that a majority of the performance gains in ML are a result of using better hardware to run brute-force statistics with larger more complex models but the algorithms themselves have been improving at a nominal rate.
  - mellosouls 6y ago
    I'm optimistic about AGI because I see no reason for it not to be implemented (though the time-frame is a different matter). Going by the hype articles (which may be unrepresentative), we just seem to be moving faster and faster on an impressively powerful, but AGI-irrelevant train along a machine "learning" railway track and although I suspect plenty of people on the train would like to get off, the drivers and momentum are making that very difficult, as indicated in the OP article. I'm completely optimistic about AGI, just think we are allowing the excitement of the advances in Artificial Unintelligence over the last few years erroneously dominate our thinking about it - at least in the sort of papers that turn up in tech-related feeds. Again, this may be unrepresentative of the top thinkers in computer science (machine-learning/whatever). My own (layman!) opinion is that the good ideas have and will continue to come from external (or intersecting) fields, philosophy, neuroscience, etc; not computer scientists raving about the power of DeepWhatever using cloud-enabled networks.
Topolomancer 6y ago
The main take-away and problem for me is the 'arXiv dilemma', combined with shoddy scholarship: newcomers to the field regularly try to drink from the arXiv firehose and take every paper they find on there as gospel—even though it is not peer-reviewed (let's set aside the issues of peer review for that one). The quick publication cycle creates an environment that is always just about 'beating' the state of the art, but if you look closer into the reported values, you will often find a lot of questionable experimental choices. In one of my main application areas, viz. graph classification, almost none of the papers holds up (with respected to the reported performance gains) if subjected to a thorough experimental setup. This creates a dangerous environment; in the worst case, we might miss some interesting contributions because they are drowned by the noise of reviewers (here we go again!) claiming that 'It does not beat the state of the art, so it must be crap'.
- dekhn 6y ago
  I think this is a main reason that journal clubs exist in grad school: to refine the ability of PhDs to separate the bullshit from the real. I was shocked at how easy it was to "take down" a paper based on straightforward errors in the methods or analysis.
- YeGoblynQueenne 6y ago
  >> In one of my main application areas, viz. graph classification, almost none of the papers holds up (with respected to the reported performance gains) if subjected to a thorough experimental setup. Could you give an example? Just being curious :)
  - Topolomancer 6y ago
    Sure thing! For the GIN-ε (https://arxiv.org/pdf/1810.00826.pdf https://arxiv.org/pdf/1810.00826.pdf), for example, the authors report a classification accuracy of 75.9±3.8 on the PROTEINS data set (classical graph benchmark data set). If you run it with a cross-validation setup that is repeated to account for effects of chance, performance drops to 73.1±0.7. Notice the drop—the second accuracy value is at least within the standard deviation of the first one, but you can see that a different experimental setup shrinks the gains quite a lot... Same goes for different data sets. Since the gains are not super large for most papers, these changes matter a lot. But of course, the paper is now published, so no one is going to go back and change it. FWIW: I like the GIN paper and think the authors did a good job. It's just that their experimental setup is insufficiently thorough for the data sets they are considering, thus leading to overoptimistic estimates. This is a problem because the next 'state of the art' paper has to find a way to get a slightly higher mean accuracy, at the expense of an even larger standard deviation, etc.
duaoebg 6y ago
My group of hard core ML friends went fully private. I don’t even pay attention to the public ML discourse (except on HN.) I think a big part of it is a low barrier to an international culture and academia which can be toxic in their own ways.
- asdff 6y ago
  Why go private?
twsttest 6y ago
"the way Yann LeCun talked about biases and fairness topics was insensitive" Insensitive according to who? The most sensitive 5% of people? All statements will be deemed insensitive by at least one person somewhere. It's silly to allow the most extremely (often unreasonably) sensitive people to set the threshold for what is sensitive or insensitive speech.
- umvi 6y ago
  > It's silly to allow the most extremely (often unreasonably) sensitive people to set the threshold for what is sensitive or insensitive speech. Well, that's one of the drawbacks of social media. Offended people can band together, amplify their voices, and spark nation-wide outrage. Whether the outrage is "real" or just "perceived" (i.e. the media says everyone is outraged so it must be so) is a different debate.
- Ar-Curunir 6y ago
  By many other researchers in the field.
- chomp 6y ago
  Insensitive to anyone who has a moderate amount of understanding of machine learning and social empathy. You can't plug your ears and say "it's just your training set" as a response to unfairness in ML algorithms. Real life is biased. Any real life data in our world is going to be biased. If you train algorithms on this data, they will cement any existing divides in society. So, with the understanding that researchers need to be more circumspect about ML algorithms than worrying about just the training data, consider that the upsampling algorithm in question only worked for white people because they fed it a huge amount of white faces. Claiming "it's just the training data" is one of those "well yes, but actually no" situations where ML researchers tend to miss the broader picture of how ML algorithms are used in real life, and just makes Yann look ignorant.
  - twsttest 6y ago
    I'm not disagreeing with your basic idea, but it seems you're nitpicking and talking past Yann's point. A model's only link to the real world is the training data, so saying it's sufficient to "worry about the training data" captures all the concerns we may have about bias, because from the model's POV there is no other relevant interface with the real world. Saying "we need to do more" is devoid of meaning when by addressing the training data we are truly doing all we can as model builders and trainers.
mrkeen 6y ago
> everybody is under attack, but nothing is improved.
bioinformatics 6y ago
The ____ (insert scientific community/field here) has a toxicity problem. It can be said from every single field in science (I am in the natural/health ones): - peer review is broken - check - reproducibility problem - check (try to reproduce any miracle cancer cure paper) - worshipping problem - check, there are kings and no one can take them down. - diversity problem - check. In my department 80% of the professors and PhD and postdocs are women. - moral and ethics are set arbitrarily - check. Morals? in Genetics? give me a brake - there is a cut-throat publish-or-perish mentality - check, tell me something new - discussions have become disrespectful - check. The time I saw Pavel Pezner giving a keynote lecture at ISMB and instead of showing his work he spend 70% of the time dressing down other people and trashing their work, Science died to me. And this was early 2000’s. Machine learning won’t be the first field to notice it, and won’t be the last. Science is not scientific anymore.
- bumby 6y ago
  How would you rephrase it? Does “science have a toxicity problem?” Academia? Humans in general?
  - asdfman123 6y ago
    Maybe we can blame the anonymity of the internet.
  - nbardy 6y ago
    Eric Weinstein has a theory that it's caused by the embedded growth obligations in our institutions. I'm not sure it's the answer, but I think there is something seriously amiss with our institutions that causes them to all have a similar set of self silencing momentum. The amount of corruption that has been exposed in institutions over the last 20 years that has gone largely ignored in traditional media and inside this institutions themselves is staggering. Every time we hear institution X has a problem with Y, the story quietly disappears and there is no institutional change enacted. https://bigthink.com/culture-religion/eric-weinstein-intellectual-dark-web?rebelltitem=4#rebelltitem4 https://bigthink.com/culture-religion/eric-weinstein-intelle...
cheesecracker 6y ago
I think as long as no government funding is involved, people sho0uld be free to worship whoever they want and read whatever they want and flock around posters of whatever topic they want. If government money is being distributed, things become more interesting. But I don't think everybody is entitled to a career funded by tax payer money. So it should probably remain "cut throat" to get a good job based on government money. Whether governments have the best criteria for handing out money is another question (number of papers published might not be the best metric).
mendelmaleh 6y ago
The west has a fragility problem.
- moccajoghurt 6y ago
  We cannot handle the freedom we had since the 90's. We abolished most rules that a conservative / christian society had. As it turns out we actually prefer having strict rules. These new rules are now about diversity, discrimination and racism. They fulfill the same role as the rules of the conservative / christian society. If you abide the rules, you are a good human being. If you are ever in doubt about yourself, just stick to the rules and you will be fine. I personally don't really like this trend but I think our society is not ready to handle freedom yet.
- asdff 6y ago
  The world has a bigotry problem.
eNTi 6y ago
The problem is not toxisity itself. That's but a symptom. It's: 1. People getting into positions for the wrong reasons (diversity hires), 2. No one has any kind of backbone any more... victim culture and molycoddling shows it's ugly face, 3. Real smart people that don't' want to content with stupid getting rightfully angry with their subpar collegues who are desperatly trying to being somewhat relevant. Let's face it... scientists are most of the time not people persons. Most smart people don't care for your snowflake persona getting hurt by the truth. It's basically impossible to do antying in an environment that treats microaggressions as anything else but the completely infantile bullshit that they are. GROW A FUCKING SPINE AND GROW THE FUCK UP WHINERS.
SmokeyHamster 6y ago
>Fifthly, machine learning, and computer science in general, have a huge diversity problem. At our CS faculty, only 30% of undergrads and 15% of the professors are women. Yes, and the deep sea fishing, oil drilling and logging industries also aren't super diverse and have a severe lack of women. I don't see anyone complaining about that. Different groups have different preferences. I've met several women who have gone into science and IT only to find it immensely unfulfilling, as it's often a very socially isolating job by nature, and leave to switch careers. If there's a rule or law preventing women from getting into the industry, then let us know and we'll change that. But don't criticize an entire industry because women on average chose to pursue other passions. >At this very moment, thousands of Uyghurs are put into concentration camps based on computer vision algorithms invented by this community, and nobody seems even remotely to care. What does he propose be done about this? Tell Chinese government bureaucrats to stop "stealing" publicly accessible research papers and code to implement tools that help commit genocide? Sue the Chinese government for violating licensing agreements that require "no violation of human rights"? Not everything should, or can be, about global politics. We should let people researching machine learning worry about machine learning, and leave the broader socio-political effects to political pundits and sociologists.
Antoninus 6y ago
It is unfortunate that culture politics have taken over HN.
chillee 6y ago
Also see: r/machinelearning has a toxicity problem https://www.reddit.com/r/MachineLearning/comments/hkc697/d_rmachinelearning_not_just_twitter_has_a https://www.reddit.com/r/MachineLearning/comments/hkc697/d_r...

143 points by zekrioca 6y ago

Agree with this point: Sixthly, moral and ethics are set arbitrarily. The U.S. domestic politics dominate every discussion. At this very moment, thousands of Uyghurs are put into concentration camps based on computer vision algorithms invented by this community, and nobody seems even remotely to care. Adding a "broader impact" section at the end of every people will not make this stop. There are huge shitstorms because a researcher wasn't mentioned in an article. Meanwhile, the 1-billion+ people continent of Africa is virtually excluded from any meaningful ML discussion (besides a few Indaba workshops).

LatteLazy 6y ago
Maybe I am misunderstanding but... Why does a community need a moral stance on external issues? If everyone is there to do/learn/advance machine learning, isn't that the end of it? It isn't their purpose to engage with Chinese or other issues. Edit: I think it makes sense to have a stance on Machine Learning moral issues in a machine learning group. I just don't think you need to have a stance on every issue ever. I say this as someone always a bit frustrated that HN and Reddit etc are very US centric.
- fennecfoxen 6y ago
  If a community has a stance on US politics which interact with it only incidentally — which the community very much does have, to the point of distraction, it is alleged — then the community can damn well afford to have a stance on international politics when the advancements the community builds are key tools in building the engines of tyrannical repression.
  - LatteLazy 6y ago
    I think it's fine to take a stance on things central to the groups main function: don't supply ML Tech for immoral use. But beyond that? That's a lot of issues you need to make stances on before you can actually do any machine learning work...
    - SpicyLemonZest 6y ago
      This issue isn't beyond that. The question is how a commitment to not supply ML tech for immoral use is compatible with supplying it to a country that's genociding the Uighurs.
    - crispyambulance 6y ago
      > don't supply ML Tech for immoral use. But beyond that? There's a lot to that and it goes beyond the scope of what researchers can handle by themselves. It's an effort that involves more than just the researchers, though they will take a central role. For one thing, how do you evaluate whether a usage is "immoral"? How do you enforce correct usage? These are difficult questions with multifaceted answers which no one has yet elucidated (yes, it has to be more than "I'll know it when I see it").
      - LatteLazy 6y ago
        This is exactly my point. You can get general agreement to "do no evil". But every step you take to be specific makes the amount of discussion and detail exponentially more difficult. So you can't have a specific stance on everything. You either take a specific stance on a small number of issues central to the community or a general stance, or you spend all your time creating and amending stances and do nothing else.
- andyjohnson0 6y ago
  > Maybe I am misunderstanding but... Why does a community need a moral stance on external issues? If everyone is there to do/learn/advance machine learning, isn't that the end of it? It isn't their purpose to engage with Chinese or other issues. Because its wrong to write-off the mistreatment of other human beings as an external issue. It never is: we're all human and we all share the same planet.
  - LatteLazy 6y ago
    So any group anywhere needs to have a stance on every injustice everywhere? Isn't that exhausting?
    - SiempreViernes 6y ago
      You just need to take a stance on injustice classes, after that its simply classification. Surely you can muster that much moral effort?
      - LatteLazy 6y ago
        Sorry, what's a class in this context I'm more than happy to say something like "injustice everywhere is bad and we should oppose it and not let our technology be used to support it". But I feel that's a bit less than people expect?
    - crispyambulance 6y ago
      > Isn't that exhausting? Eventually, but this is a unique time right now. People are paying attention to injustice. I would say that much of these "stances" are inauthentic and done just for PR reasons. We'll see who is being genuine and who is phony soon enough.
      - protomyth 6y ago
        No, people are paying attention to some injustice. They don't give a damn about other injustices unless they can wrap those back into one of their causes. They'll visit Native Americans when its something about environment, but ignore them when it comes to police violence.
- deleuze 6y ago
  How the technology you develop is used is an "external issue"? Ivory tower much?
  - kiliantics 6y ago
    Would you say the issues are more "rhizomatic"? ;)
- crispyambulance 6y ago
  > Why does a community need a moral stance on external issues? Some communities choose to do so. Einstein, Szilard, Pauling (and others) formed the Emergency Committee of Atomic Scientists to argue against nuclear weapons at the dawn of the Atomic era. And many scientists have taken a stand against nuclear weapons since then. Their actions have, at least, influenced the proliferation of these weapons and informed the public about their dangers. Would you argue that it wasn't "their business?" One could make a case that machine learning has negative consequences/possibilities that are similar in scope to nuclear weapons. The people who work with and understand machine learning are in the best position to take a stand against abuse of these technologies. I would say it's their responsibility to speak out, otherwise a despot will do it for them.
  - 0-_-0 6y ago
    > Their actions have, at least, influenced the proliferation of these weapons How? The number of nuclear weapons was growing rapidly until the 80s, to the point where dozens of bombs were aimed at single targets. > informed the public about their dangers Seeing the effects of bombs informed the public, not a committee
    - crispyambulance 6y ago
      > How? The number of nuclear weapons was growing rapidly until the 80s. It's hard to assess their impact, but that doesn't mean it was insignificant. Could we have been in a worse position? Definitely. Someone might have "pushed the button" by now were it not for decades of work from organizations (some of whom consisted of scientists) lobbying against nuclear weapons and their proliferation. > Seeing the effects of bombs informed the public, not a committee The nuclear tests demonstrated that the weapons work. There are other effects such as "nuclear winter" that have not been (and hopefully never will be) demonstrated. Sadly, because of Chernobyl, we do have examples of what happens to contaminated areas. Understanding and communicating the effects beyond "blast radius" is best done by scientists and health experts.
- SpicyLemonZest 6y ago
  Not every community does. The ML community generally believes they're obligated to have moral stances on external issues because they believe machine learning has great potential to cause harm if not properly controlled.
  - LatteLazy 6y ago
    Thanks. I guess I can admire the intent and the recognition of consequences of important tech. Honestly though, I'm glad I'm not responsible for organising that sort of response...
- whytaka 6y ago
  The world is changing vigorously. Trust is breaking down domestically and internationally. For some, it was never trust but an oppressive stability. People are anxious about who they are dealing with because who rises to power now will determine which subcultures will enjoy great political prominence and who will have their voices silenced in the near future. The battle is over values, standards, and world views. A lot is at stake. Technology is the ultimate medium now. Technologies like machine learning shapes discussions like money and GDP figures continues to shape discussions about the state of societies the world over. Ignore the strategically important position that technology holds now and in the future of our society at your own peril. To be so willfully blind to the changing times, I can’t imagine a less engaged worldview.
- michaelmrose 6y ago
  If your research is used to create such ill that on net throwing every researcher in your field into the sea would be a lesser evil the question of external issues at least deserves a look lest you miss an opportunity to avert harm. Note I do not support tossing ml researchers into the sea and lots of industries are worse.
Udik 6y ago
> thousands of Uyghurs are put into concentration camps based on computer vision algorithms invented by this community I saw this repeated frequently in the last few days, and I think I've missed some important info. I know that China uses face recognition for surveillance purpose, and that it detains a substantial share of the Uighur population for supposed violations of the law. I miss a link between the two: is China using AI to detect who is Uighur and arrest him/ her for this very fact? Does anyone have more info?
- NikolaeVarius 6y ago
  https://www.nytimes.com/2019/04/14/technology/china-surveillance-artificial-intelligence-racial-profiling.html https://www.nytimes.com/2019/04/14/technology/china-surveill...
  - Udik 6y ago
    Thank you, but this article seems to say that the technology is used to track Uighurs (I suppose for the reason that some of them have been engaging in terrorist activities) and not to incarcerate them. Despite the scale- which we considered worrying in itself and for understandable reasons- this seems to be an automated version of "I suddenly see a lot of mafioso-looking guys congregating here, let's keep an eye to see if there's anything dodgy going on"- which has been practiced with skilled eyes since forever. And it's a whole different thing from "if I see anyone looking Jewish I'll round them up and send them to a concentration camp".
    - cycomanic 6y ago
      > Thank you, but this article seems to say that the technology is used to track Uighurs (I suppose for the reason that some of them have been engaging in terrorist activities) and not to incarcerate them. That's a weird suggestion. It's undisputed that they have put millions into detention camps, so are you suggesting they all have been engaging in "terrorist activities"? So why suggest ("suppose" ) that they have been engaging in terrorist activities? > Despite the scale- which we considered worrying in itself and for understandable reasons- this seems to be an automated version of "I suddenly see a lot of mafioso-looking guys congregating here, let's keep an eye to see if there's anything dodgy going on"- which has been practiced with skilled eyes since forever. And it's a whole different thing from "if I see anyone looking Jewish I'll round them up and send them to a concentration camp". The article literally said they are looking for (tracking) Uighurs because of their ethnic appearance. That is looking for "Jewish" , not looking "mafioso-looking" in your example above

craftinator 6y ago

I would argue that anything that becomes sufficiently popular enough to be lucrative beyond some arbitrary threshold becomes toxic. Welcome to the cycle of capitalism!

godelzilla 6y ago
Progress < profits and control
- DudeInBasement 6y ago
  Just a series of weighted if statements anyway...

tomp 6y ago

> At our CS faculty, only 30% of undergrads and 15% of the professors are women. Going on parental leave during a PhD or post-doc usually means the end of an academic career. However, this lack of diversity is often abused as an excuse to shield certain people from any form of criticism. Reducing every negative comment in a scientific discussion to race and gender creates a toxic environment. People are becoming afraid to engage in fear of being called a racist or sexist, which in turn reinforces the diversity problem. This was an unexpected twist! It's rare to read such an honest, unbiased opinion on this issue.

deleuze 6y ago
I'm glad you've honed in to identify that the 15-30% of women in machine learning are actually the real problem. Which seems consistent with your previous claim that: > I'm a man, but I've never received any benefit just because (most of) our presidents / businessmen are men (in fact, arguably (individual) women receive more benefits, because most of those men are heterosexuals). A small minority of women are actually the real privileged ones, huh?
- new2628 6y ago
  They are not the real problem. The all-or-nothing exaggerations that remove all nuance are (a major part of) the real problem.
  - deleuze 6y ago
    I think the person saying that "akshually, women are more privileged because I'm a heterosexual" is the one lacking nuance.
    - SpicyLemonZest 6y ago
      Nobody said this; you're inventing strawmen. e: Apparently someone did say this. Apologies.
      - 6y ago

mellosouls 6y ago

This is very refreshing to read - looking from the outside in, there has been a pretty clear problem with (at least the representation of) ML research along the lines highlighted here - ridiculous hype, papers again and again linked from the same researchers and institutions indicating potential cliqueyness, nepotism and celebrity worship. All very dispiriting, indicating real problems making progress in AGI beyond the clear lack of ideas. Of course, just because these toxicity claims are being made, it doesn't mean they are accurate, but they certainly ring true. If they are, it is good to know they are being talked about in some quarters.

foolmeonce 6y ago
> Of course, just because these toxicity claims are being made, it doesn't mean they are accurate, but they certainly ring true. In many respects I think it is more important that we have no idea what is true about any industry. When I dwell too much about my impression of any part of the tech field I usually end up demotivated to do anything that might connect me with the social group involved. I think there is a general shallowness and overcompensation in our industry that makes it impossible to figure out what real life is like with a group until you are way too invested and they have already hazed you.
zippy5 6y ago
I'm super curious why you were optimistic about AGI in the first place? it seem to me that a majority of the performance gains in ML are a result of using better hardware to run brute-force statistics with larger more complex models but the algorithms themselves have been improving at a nominal rate.
- mellosouls 6y ago
  I'm optimistic about AGI because I see no reason for it not to be implemented (though the time-frame is a different matter). Going by the hype articles (which may be unrepresentative), we just seem to be moving faster and faster on an impressively powerful, but AGI-irrelevant train along a machine "learning" railway track and although I suspect plenty of people on the train would like to get off, the drivers and momentum are making that very difficult, as indicated in the OP article. I'm completely optimistic about AGI, just think we are allowing the excitement of the advances in Artificial Unintelligence over the last few years erroneously dominate our thinking about it - at least in the sort of papers that turn up in tech-related feeds. Again, this may be unrepresentative of the top thinkers in computer science (machine-learning/whatever). My own (layman!) opinion is that the good ideas have and will continue to come from external (or intersecting) fields, philosophy, neuroscience, etc; not computer scientists raving about the power of DeepWhatever using cloud-enabled networks.

Topolomancer 6y ago

The main take-away and problem for me is the 'arXiv dilemma', combined with shoddy scholarship: newcomers to the field regularly try to drink from the arXiv firehose and take every paper they find on there as gospel—even though it is not peer-reviewed (let's set aside the issues of peer review for that one). The quick publication cycle creates an environment that is always just about 'beating' the state of the art, but if you look closer into the reported values, you will often find a lot of questionable experimental choices. In one of my main application areas, viz. graph classification, almost none of the papers holds up (with respected to the reported performance gains) if subjected to a thorough experimental setup. This creates a dangerous environment; in the worst case, we might miss some interesting contributions because they are drowned by the noise of reviewers (here we go again!) claiming that 'It does not beat the state of the art, so it must be crap'.

dekhn 6y ago
I think this is a main reason that journal clubs exist in grad school: to refine the ability of PhDs to separate the bullshit from the real. I was shocked at how easy it was to "take down" a paper based on straightforward errors in the methods or analysis.
YeGoblynQueenne 6y ago
>> In one of my main application areas, viz. graph classification, almost none of the papers holds up (with respected to the reported performance gains) if subjected to a thorough experimental setup. Could you give an example? Just being curious :)
- Topolomancer 6y ago
  Sure thing! For the GIN-ε (https://arxiv.org/pdf/1810.00826.pdf https://arxiv.org/pdf/1810.00826.pdf), for example, the authors report a classification accuracy of 75.9±3.8 on the PROTEINS data set (classical graph benchmark data set). If you run it with a cross-validation setup that is repeated to account for effects of chance, performance drops to 73.1±0.7. Notice the drop—the second accuracy value is at least within the standard deviation of the first one, but you can see that a different experimental setup shrinks the gains quite a lot... Same goes for different data sets. Since the gains are not super large for most papers, these changes matter a lot. But of course, the paper is now published, so no one is going to go back and change it. FWIW: I like the GIN paper and think the authors did a good job. It's just that their experimental setup is insufficiently thorough for the data sets they are considering, thus leading to overoptimistic estimates. This is a problem because the next 'state of the art' paper has to find a way to get a slightly higher mean accuracy, at the expense of an even larger standard deviation, etc.

duaoebg 6y ago

My group of hard core ML friends went fully private. I don’t even pay attention to the public ML discourse (except on HN.) I think a big part of it is a low barrier to an international culture and academia which can be toxic in their own ways.

asdff 6y ago
Why go private?

twsttest 6y ago

"the way Yann LeCun talked about biases and fairness topics was insensitive" Insensitive according to who? The most sensitive 5% of people? All statements will be deemed insensitive by at least one person somewhere. It's silly to allow the most extremely (often unreasonably) sensitive people to set the threshold for what is sensitive or insensitive speech.

umvi 6y ago
> It's silly to allow the most extremely (often unreasonably) sensitive people to set the threshold for what is sensitive or insensitive speech. Well, that's one of the drawbacks of social media. Offended people can band together, amplify their voices, and spark nation-wide outrage. Whether the outrage is "real" or just "perceived" (i.e. the media says everyone is outraged so it must be so) is a different debate.
Ar-Curunir 6y ago
By many other researchers in the field.
chomp 6y ago
Insensitive to anyone who has a moderate amount of understanding of machine learning and social empathy. You can't plug your ears and say "it's just your training set" as a response to unfairness in ML algorithms. Real life is biased. Any real life data in our world is going to be biased. If you train algorithms on this data, they will cement any existing divides in society. So, with the understanding that researchers need to be more circumspect about ML algorithms than worrying about just the training data, consider that the upsampling algorithm in question only worked for white people because they fed it a huge amount of white faces. Claiming "it's just the training data" is one of those "well yes, but actually no" situations where ML researchers tend to miss the broader picture of how ML algorithms are used in real life, and just makes Yann look ignorant.
- twsttest 6y ago
  I'm not disagreeing with your basic idea, but it seems you're nitpicking and talking past Yann's point. A model's only link to the real world is the training data, so saying it's sufficient to "worry about the training data" captures all the concerns we may have about bias, because from the model's POV there is no other relevant interface with the real world. Saying "we need to do more" is devoid of meaning when by addressing the training data we are truly doing all we can as model builders and trainers.

mrkeen 6y ago

> everybody is under attack, but nothing is improved.

bioinformatics 6y ago

The ____ (insert scientific community/field here) has a toxicity problem. It can be said from every single field in science (I am in the natural/health ones): - peer review is broken - check - reproducibility problem - check (try to reproduce any miracle cancer cure paper) - worshipping problem - check, there are kings and no one can take them down. - diversity problem - check. In my department 80% of the professors and PhD and postdocs are women. - moral and ethics are set arbitrarily - check. Morals? in Genetics? give me a brake - there is a cut-throat publish-or-perish mentality - check, tell me something new - discussions have become disrespectful - check. The time I saw Pavel Pezner giving a keynote lecture at ISMB and instead of showing his work he spend 70% of the time dressing down other people and trashing their work, Science died to me. And this was early 2000’s. Machine learning won’t be the first field to notice it, and won’t be the last. Science is not scientific anymore.

bumby 6y ago
How would you rephrase it? Does “science have a toxicity problem?” Academia? Humans in general?
- asdfman123 6y ago
  Maybe we can blame the anonymity of the internet.
- nbardy 6y ago
  Eric Weinstein has a theory that it's caused by the embedded growth obligations in our institutions. I'm not sure it's the answer, but I think there is something seriously amiss with our institutions that causes them to all have a similar set of self silencing momentum. The amount of corruption that has been exposed in institutions over the last 20 years that has gone largely ignored in traditional media and inside this institutions themselves is staggering. Every time we hear institution X has a problem with Y, the story quietly disappears and there is no institutional change enacted. https://bigthink.com/culture-religion/eric-weinstein-intellectual-dark-web?rebelltitem=4#rebelltitem4 https://bigthink.com/culture-religion/eric-weinstein-intelle...

cheesecracker 6y ago

I think as long as no government funding is involved, people sho0uld be free to worship whoever they want and read whatever they want and flock around posters of whatever topic they want. If government money is being distributed, things become more interesting. But I don't think everybody is entitled to a career funded by tax payer money. So it should probably remain "cut throat" to get a good job based on government money. Whether governments have the best criteria for handing out money is another question (number of papers published might not be the best metric).

mendelmaleh 6y ago

The west has a fragility problem.

moccajoghurt 6y ago
We cannot handle the freedom we had since the 90's. We abolished most rules that a conservative / christian society had. As it turns out we actually prefer having strict rules. These new rules are now about diversity, discrimination and racism. They fulfill the same role as the rules of the conservative / christian society. If you abide the rules, you are a good human being. If you are ever in doubt about yourself, just stick to the rules and you will be fine. I personally don't really like this trend but I think our society is not ready to handle freedom yet.
asdff 6y ago
The world has a bigotry problem.

eNTi 6y ago

The problem is not toxisity itself. That's but a symptom. It's: 1. People getting into positions for the wrong reasons (diversity hires), 2. No one has any kind of backbone any more... victim culture and molycoddling shows it's ugly face, 3. Real smart people that don't' want to content with stupid getting rightfully angry with their subpar collegues who are desperatly trying to being somewhat relevant. Let's face it... scientists are most of the time not people persons. Most smart people don't care for your snowflake persona getting hurt by the truth. It's basically impossible to do antying in an environment that treats microaggressions as anything else but the completely infantile bullshit that they are. GROW A FUCKING SPINE AND GROW THE FUCK UP WHINERS.

SmokeyHamster 6y ago

>Fifthly, machine learning, and computer science in general, have a huge diversity problem. At our CS faculty, only 30% of undergrads and 15% of the professors are women. Yes, and the deep sea fishing, oil drilling and logging industries also aren't super diverse and have a severe lack of women. I don't see anyone complaining about that. Different groups have different preferences. I've met several women who have gone into science and IT only to find it immensely unfulfilling, as it's often a very socially isolating job by nature, and leave to switch careers. If there's a rule or law preventing women from getting into the industry, then let us know and we'll change that. But don't criticize an entire industry because women on average chose to pursue other passions. >At this very moment, thousands of Uyghurs are put into concentration camps based on computer vision algorithms invented by this community, and nobody seems even remotely to care. What does he propose be done about this? Tell Chinese government bureaucrats to stop "stealing" publicly accessible research papers and code to implement tools that help commit genocide? Sue the Chinese government for violating licensing agreements that require "no violation of human rights"? Not everything should, or can be, about global politics. We should let people researching machine learning worry about machine learning, and leave the broader socio-political effects to political pundits and sociologists.

Antoninus 6y ago

It is unfortunate that culture politics have taken over HN.

chillee 6y ago

Also see: r/machinelearning has a toxicity problem https://www.reddit.com/r/MachineLearning/comments/hkc697/d_rmachinelearning_not_just_twitter_has_a https://www.reddit.com/r/MachineLearning/comments/hkc697/d_r...