8 ms·

The NeurIPS 2020 broader impacts experiment

58 points by luu 6y ago

echelon 6y ago
> In order to provide a balanced perspective, authors are required to include a statement of the potential broader impact of their work, including its ethical aspects and future societal consequences. Authors should take care to discuss both positive and negative outcomes Well Google, there you go.
- trhway 6y ago
  I wonder whether/when "ethical AI" is going to metastasize into other areas of software engineering as well as into sciences, like physics, biology, chemistry. I mean is NN research is more dangerous than say nuclear or is it just self-aggrandizing ?
  - maxov 6y ago
    Uh, yes? As just one example, China is already using facial recognition technology to help them identify, control, and jail Uyghurs: https://www.npr.org/2019/12/16/788597818/how-china-is-using-facial-recognition-technology https://www.npr.org/2019/12/16/788597818/how-china-is-using-...
    - trhway 6y ago
      Ethnical oppression was a thing long before, not sure what facial recognition changes here.
      - maxov 6y ago
        I'm not sure what this implies. Nuclear is a good example, because it can have immense positives in providing clean and safe power. War has existed for millennia, but that doesn't mean nuclear weapons aren't a uniquely dangerous technology to enable it. The example I gave is perhaps the worst possible scenario, where AI systems are specifically engineered for the goal of racial bias. The recent advancements in facial recognition technology (helped by the great progress in image recognition by neural nets) make possible a wide-spread surveillance system. I would certainly call the scale enabled by the automatable human-level performance of facial recognition systems dangerous, just like I would call the scale of destruction of nuclear weapons dangerous, even if they are dangerous in different ways. Even then, e.g. in the United States AI systems aren't necessarily engineered for bias. However, if we're not careful, existing racial, gender, or socioeconomic biases in society can enter these systems and can be reinforced by them (this has already happened in well-documented ways). There I think AI ethics researchers can provide immense value in helping us identify and fix these urgent problems, and I welcome ethics researchers playing a part in any field for this reason.
  - throwawaygh 6y ago
    Nuclear and crypto already have extreme ethics reviews with super high stakes called "export controls" ;) I kid, but only kind of.
the_snooze 6y ago
In the security research community, it's common for reviewers to expect discussions of things like responsible disclosures, informed consent, data confidentiality, and the legality and ethics of research methods. I think it's a great thing for researchers to be cognizant of their work's broader impacts; not just for harm reduction, but because it makes your work so much more interesting to a wide variety people.
- Permit 6y ago
  I'm not familiar the security research community, is it common for papers to be rejected for the reasons you listed?
  - the_snooze 6y ago
    It hasn't been common in the program committees I've served on, and I think that's mainly a function of people generally knowing what the expectations are. I've had my own papers get review comments like "good work, but we need you to include justification for why your approach isn't illegal under the CFAA." Authors (myself included) are generally very receptive to that kind of feedback because it helps keep us out of trouble down the road. Here's an example of a disclaimer that shows the authors were acting in good faith (not mine): >As performing a security analysis against a running election server would raise a number of unacceptable legal and ethical concerns, we instead chose to perform all of our analyses in a "cleanroom" environment, connecting only to our own servers. Special care was taken to ensure that our static and dynamic analysis techniques could never interfere with Voatz or any related services, and we went through great effort so that nothing was intentionally transmitted to Voatz’s servers. https://www.usenix.org/system/files/sec20-specter.pdf https://www.usenix.org/system/files/sec20-specter.pdf
- moyix 6y ago
  This is true – but at the same time the security research community doesn't require that every paper do this, only that if your paper has potential for harm that you discuss how it was managed. It definitely comes up in program committee meetings though (and it tends to provoke some of the most heated discussions).
linspace 6y ago
Science is objective and there is a long history of morality and ethical changes, and incorrect evaluation to societal impact. I think this is a concesion to political pressure.
- AnHonestComment 6y ago
  It’s 100% political. Like shadowbanning me for being an “inherent flame war”.
- marcinzm 6y ago
  >Science is objective Nothing done by humans is objective. Objective systems don't, for example, have massive reproducibility problems.
  - linspace 6y ago
    Science cannot be trusted, right?
    - marcinzm 6y ago
      Please don't put words in my mouth. I simply said it is not objective. You can trust non-objective things and derive value from them. However in the end they are still biased.
  - unishark 6y ago
    > Nothing done by humans is objective. This seems like a hyperbole. Obviously there is a lot of garbage that people claim to be science. But 2+2=4 is subjective?
    - kaitai 6y ago
      We are still humans interacting with and representing science/mathematics. Sure, 2+2=4, unless you're working base 3 or something, in which case you just have to be careful about your definitions. An easy caveat in hindsight but too often people take the truth they know and insist it's the only one. Just look at the debate over irrational numbers in the time of the Greeks -- they can't exists, claims one side, and as evidence they note that they can't represent them. Or imaginary numbers: they too can't exist. Or parallel lines that cross (what?). Or breaking the rules of calculus to get generalized functions. Basically one thing I love about math is you can take any statement/idea/construction, consider some opposite of it, and see if you can get something coherent out -- and often enough you do! I'm not saying that 2+2=4 or "you can't differentiate discontinuous functions" is subjective, per se, but it is in some sense a decision, a choice.
bra-ket 6y ago
> 'Regardless of scientific quality or contribution, a submission may be rejected for ethical considerations, including methods, applications, or data that create or reinforce unfair bias' In other words science has to pass trough political filter of the day. If reality is wrong, unfair or politically incorrect (in our subjective view), we're going to reject reality. Prof. Pedro Domingos published a great piece on this https://spectator.us/militant-liberals-politicizing-artificial-intelligence/ https://spectator.us/militant-liberals-politicizing-artifici...
- Der_Einzige 6y ago
  I think your vitriol is wrong, but there is a kernel of truth. AI with no debiasing is a mirror. It's pattern recognition. Techniques for achieving fair and equitable AI are frequently quite similar to "playing god" and do come dangerously close to implementing "rejecting your reality and substituting my own" into the data through sophisticated techniques for social justice reasons. This can elevate the impact of an AI developers actions far above what they may have expected. For instance, I am shocked that the right wing media has not reported more about the "censorship" inherent to some kinds of word vector debiasing techniques proposed in papers after the famous "man is to computer programmer as woman is to homemaker?" paper within NLP. Usually the undesirable associations (programmer is "male" and homemaker is "female") are due to a representation gap either inherit to reality (far fewer female programmers) or due to failure to properly write about and represent female programmers in corpra (Wikipedia editors should write more articles about female programmers). One of the techniques proposed to prevent this bias is to simply find the specific rows in your dataset that maximizes the undesirable association and remove these rows. In practice, most of these "undesirable" rows (let's say a row is a wikipedia article) will have no inherit bias what-so-ever, and only get their poor scores because the article is about someone where the editors/authors wrote a lot about their relationship with their dad (using male words) before talking about their programming career. It leaves me with a bad taste in my mouth to remove not-sexist articles to mitigate or remove sexist associations. Maybe the OP could have critiqued this kind of work rather than their original post and it may have been more constructive...
  - 6y ago
Permit 6y ago
> Only four papers were rejected because of ethical considerations, after a thorough assessment that included the original technical reviewers, the area chair, the senior area chair and also the program chairs. Is it possible to read these papers?
- CJefferson 6y ago
  In general the rejected papers are never discussed by conferences, as the authors will want to keep their from being public until they can publish it. I don't entirely agree with this, and some subjects are moving away from this way of working, but it is still very common in computing.
  - dhairya 6y ago
    I agree as well, it's valuable to see rejected papers or at the very least the reviews themselves. It's interesting that some Neurips workshops are on OpenReview but Neurips itself is not. I've found great value in being able to read the reviews (both as a reviewer and learning how to do good reviews and as writer)
    - craigacp 6y ago
      The reviews for published NeurIPS papers for at least the past 5 years are available from each paper's page on http://papers.neurips.cc http://papers.neurips.cc.
nharada 6y ago
I for one am happy to see this. I often see the argument that technology can be good or bad depending on how it's used, and I would agree with this. However, too often the solution to preventing misuse proposed by people is "don't think about it", which doesn't seem like a good one to me.
Master_Odin 6y ago
A lot of these pieces on "ethical AI" just seems like theater to me. I'm sure if you ask the authors of the modern techniques of facial recognition, they'd tell you all about the wonderful and great things you can accomplish with it. And yet we also have state actors using it for great harm, and this is true of a lot of foundational AI work that then powers AI solutions to problems. Maybe this kind of "push" from conferences will help power a change, but the given the replication crisis (which the author mentions) has been well known for a long time and nothing has really happened on that front, I wouldn't hold my breath.
- asciident 6y ago
  There's a handful of ethical AI tropes I hear over and over. The (made up) story about Target predicting a pregnancy before she knew, the concern of using it to predict parole, face detection not working with dark-skinned people, and the trolley problem. Every talk is some permutation of these. I would love to hear actual solutions, guidelines, and deeper threads of research.
  - naresh_xai 6y ago
    Happy to share a lot more ethical and challenges of non transparent AI via email with you. Bigger companies are pretty much hiding behind non transparent models and trying to ignore their failures at the moment.
tawtawtaw 6y ago
This is a theory-focused academic conference, so all of the work being presented is pretty fundamental, meaning it could underpin all kinds of applications. By analogy, if I invent a hammer that is balanced to drive nails faster, should I consider the broader impact that a bad actor could build a shelter faster? This is all about politicizing fundamental research, because it's in a field that's in the spotlight. As work becomes more applied, I can see the logic that scrutiny and consideration of its ethical implications should be increased, but pretending that authors of the kind of fundamental advances presented at a research conference like this need to speculate on the kind of end applications they could underpin is just people with nothing of value to contribute trying to muscle in to a popular venue.
- kaitai 6y ago
  I was a theoretical mathematician for a long time, and even there we had to talk about Broader Impacts for NSF grants, for instance. It wasn't really talking about applications for us, necessarily -- people touted the summer camps they'd present at or the undergrads they'd sponsor for research or the way they would give grant money to help undergrads attend conferences (always tied to the pure math in the grant). These NeurIPS Broader Impact statements seem a bit different, in that they more explicitly call out ethical considerations. It's funny though that you ask about hammers & nails. Pure math had a strong tradition of people ending up there because they did not want to do applied math, physics, or engineering that could be weaponized and used to kill people/support oppressive governments/infringe upon basic freedoms. Pure math is not particularly thoughtful about ethics, but many folks ended up there because of their ethical views. The Cold War, World War 2, Vietnam, and French occupation of Algeria had particular ripple effects in pure math because of that. It would be a shame if ML also became a field people avoided because they didn't want to contribute to evil, in their own view. Take the longer view: it's easy to think we're the first people ever to think about ethics and science/tech/math, but far from it. How are today's struggles the same or different than Grothendieck's, or Laurent Schwartz's, or Emmy Noether's, or Lagrange's or Kovalevskaya's or...? They all had significant reckonings to contend with.
  - mkasu 6y ago
xiphias2 6y ago
While it's great to talk about broader impact of new algorithms, I think it's too late to limit AI from taking control over humanity. I see exponentially stronger algorithms every year, but the advancement in being able to set a limit to it is nothing so far.
- croissants 6y ago
  > I see exponentially stronger algorithms every year You do? I don't. My sense as somebody near (but not in) the ML research community is that, while there have been recent flashy new things like GPT3 or the new protein folding benchmark, these are more like "solid improvements to applications of things we mostly already know". Are you referring to something else?
  - xiphias2 6y ago
    You can take almost any deep learning task that we had 5 years ago, and we can train the same task about 1000x cheaper because of hardware improvements that are slowing down, and algorithmic improvements that don't seem to be slowing down. Jeff Dean had an overview article about it I think. Also many people forget that we have working self driving cars on the road, 20 years ago nobody thought that it would happen so fast.
lasagnaphil 6y ago
The more important issue with ethics in AI is that there aren't that many people well-versed in both technology and philosophy (Throughout history many prominent scientists and engineers also dabbled a lot in the humanities, but today it seems there aren't that much.) Therefore, on one side many people in the humanities greatly misunderstands and either fantasizes or fear-mongers AI up to ridiculous heights (sometimes venturing into science-fiction territory), or many scientists and the engineers think there is absolutely no problem in applying algorithms that can be comparable to phrenology [1] to society. But most importantly, it seems like there needs to be a fundamental advancement in philosophy (beyond the current state of modernism and postmodernism), if we really want to criticize the negative aspects of technocratic governance. Maybe here's an idea I had for a while: I think there is a deeper lineage to machine learning that nobody is talking about: optimization. (Hot take: isn't what we're all doing with deep learning just optimization to fit large datasets?) I think in order to actually create a meaningful discussion about today's AI, it is crucial to look back at the history of optimization (all the way back into the 30s Soviet, where they tried to use linear programming to their planned economy and failed [2].) I think there needs to be a philosophical framework capable of articulating the effects of optimization technology on our society, before we can go any further and tackle the issues with today's and tomorrow's AI. [1] https://en.wikipedia.org/wiki/Phrenology https://en.wikipedia.org/wiki/Phrenology [2] https://crookedtimber.org/2012/05/30/in-soviet-union-optimization-problem-solves-you/ https://crookedtimber.org/2012/05/30/in-soviet-union-optimiz...
Fede_V 6y ago
I'd rather have a section of the conference that's clearly dedicated to ethical issues in AI - or - mandating rigorous discussion in papers where it is clearly warranted (ie, anything concerning facial recognition/generation, de-identification, etc) than asking all authors to write an additional section. NeurIPS is an extraordinarily competitive conference. Most authors will look at it as another way to potentially be rejected by a reviewer/area chair (in spite of all assurances about the process), and will write something incredibly trite and bland to make sure nobody is upset. Another concern is that this leads to a death by a thousand cuts - it's very easy to justify asking authors to write an additional section - but - if someone with sufficient authority doesn't say no, the logical endpoint is something like the application process for tenure track positions, where you need to produce something in the order of hundreds of pages (teaching statement, research statement, cover letter, diversity statement, list of funding, recommendation letters, etc).
throwawaysea 6y ago
The injection of politics and fuzzy notions of morality into science and technology is corrupting the objective truth seeking foundations of these fields. NeurIPS needs to abandon this direction immediately or else its legitimacy will be in question. This will lead to the publishing authors affixing pandering impact statements to cater to the personal politics whims of those judging their work, and will also lead them to avoid risky exploration or challenges to the political status quo. The legitimacy of AI Ethics as a subject is in serious doubt already after the Timnit Gebru fiasco, but if NeurIPS wants to invite politics into the AI world, they should invite those writing papers on ethics to analyze others’ work instead of requiring such impact statements of authors who are focused on core technology.

58 points by luu 6y ago

> In order to provide a balanced perspective, authors are required to include a statement of the potential broader impact of their work, including its ethical aspects and future societal consequences. Authors should take care to discuss both positive and negative outcomes Well Google, there you go.

trhway 6y ago
I wonder whether/when "ethical AI" is going to metastasize into other areas of software engineering as well as into sciences, like physics, biology, chemistry. I mean is NN research is more dangerous than say nuclear or is it just self-aggrandizing ?
- maxov 6y ago
  Uh, yes? As just one example, China is already using facial recognition technology to help them identify, control, and jail Uyghurs: https://www.npr.org/2019/12/16/788597818/how-china-is-using-facial-recognition-technology https://www.npr.org/2019/12/16/788597818/how-china-is-using-...
  - trhway 6y ago
    Ethnical oppression was a thing long before, not sure what facial recognition changes here.
    - maxov 6y ago
      I'm not sure what this implies. Nuclear is a good example, because it can have immense positives in providing clean and safe power. War has existed for millennia, but that doesn't mean nuclear weapons aren't a uniquely dangerous technology to enable it. The example I gave is perhaps the worst possible scenario, where AI systems are specifically engineered for the goal of racial bias. The recent advancements in facial recognition technology (helped by the great progress in image recognition by neural nets) make possible a wide-spread surveillance system. I would certainly call the scale enabled by the automatable human-level performance of facial recognition systems dangerous, just like I would call the scale of destruction of nuclear weapons dangerous, even if they are dangerous in different ways. Even then, e.g. in the United States AI systems aren't necessarily engineered for bias. However, if we're not careful, existing racial, gender, or socioeconomic biases in society can enter these systems and can be reinforced by them (this has already happened in well-documented ways). There I think AI ethics researchers can provide immense value in helping us identify and fix these urgent problems, and I welcome ethics researchers playing a part in any field for this reason.
- throwawaygh 6y ago
  Nuclear and crypto already have extreme ethics reviews with super high stakes called "export controls" ;) I kid, but only kind of.

the_snooze 6y ago

In the security research community, it's common for reviewers to expect discussions of things like responsible disclosures, informed consent, data confidentiality, and the legality and ethics of research methods. I think it's a great thing for researchers to be cognizant of their work's broader impacts; not just for harm reduction, but because it makes your work so much more interesting to a wide variety people.

Permit 6y ago
I'm not familiar the security research community, is it common for papers to be rejected for the reasons you listed?
- the_snooze 6y ago
  It hasn't been common in the program committees I've served on, and I think that's mainly a function of people generally knowing what the expectations are. I've had my own papers get review comments like "good work, but we need you to include justification for why your approach isn't illegal under the CFAA." Authors (myself included) are generally very receptive to that kind of feedback because it helps keep us out of trouble down the road. Here's an example of a disclaimer that shows the authors were acting in good faith (not mine): >As performing a security analysis against a running election server would raise a number of unacceptable legal and ethical concerns, we instead chose to perform all of our analyses in a "cleanroom" environment, connecting only to our own servers. Special care was taken to ensure that our static and dynamic analysis techniques could never interfere with Voatz or any related services, and we went through great effort so that nothing was intentionally transmitted to Voatz’s servers. https://www.usenix.org/system/files/sec20-specter.pdf https://www.usenix.org/system/files/sec20-specter.pdf
moyix 6y ago
This is true – but at the same time the security research community doesn't require that every paper do this, only that if your paper has potential for harm that you discuss how it was managed. It definitely comes up in program committee meetings though (and it tends to provoke some of the most heated discussions).

linspace 6y ago

Science is objective and there is a long history of morality and ethical changes, and incorrect evaluation to societal impact. I think this is a concesion to political pressure.

AnHonestComment 6y ago
It’s 100% political. Like shadowbanning me for being an “inherent flame war”.
marcinzm 6y ago
>Science is objective Nothing done by humans is objective. Objective systems don't, for example, have massive reproducibility problems.
- linspace 6y ago
  Science cannot be trusted, right?
  - marcinzm 6y ago
    Please don't put words in my mouth. I simply said it is not objective. You can trust non-objective things and derive value from them. However in the end they are still biased.
- unishark 6y ago
  > Nothing done by humans is objective. This seems like a hyperbole. Obviously there is a lot of garbage that people claim to be science. But 2+2=4 is subjective?
  - kaitai 6y ago
    We are still humans interacting with and representing science/mathematics. Sure, 2+2=4, unless you're working base 3 or something, in which case you just have to be careful about your definitions. An easy caveat in hindsight but too often people take the truth they know and insist it's the only one. Just look at the debate over irrational numbers in the time of the Greeks -- they can't exists, claims one side, and as evidence they note that they can't represent them. Or imaginary numbers: they too can't exist. Or parallel lines that cross (what?). Or breaking the rules of calculus to get generalized functions. Basically one thing I love about math is you can take any statement/idea/construction, consider some opposite of it, and see if you can get something coherent out -- and often enough you do! I'm not saying that 2+2=4 or "you can't differentiate discontinuous functions" is subjective, per se, but it is in some sense a decision, a choice.

bra-ket 6y ago

> 'Regardless of scientific quality or contribution, a submission may be rejected for ethical considerations, including methods, applications, or data that create or reinforce unfair bias' In other words science has to pass trough political filter of the day. If reality is wrong, unfair or politically incorrect (in our subjective view), we're going to reject reality. Prof. Pedro Domingos published a great piece on this https://spectator.us/militant-liberals-politicizing-artificial-intelligence/ https://spectator.us/militant-liberals-politicizing-artifici...

Der_Einzige 6y ago
I think your vitriol is wrong, but there is a kernel of truth. AI with no debiasing is a mirror. It's pattern recognition. Techniques for achieving fair and equitable AI are frequently quite similar to "playing god" and do come dangerously close to implementing "rejecting your reality and substituting my own" into the data through sophisticated techniques for social justice reasons. This can elevate the impact of an AI developers actions far above what they may have expected. For instance, I am shocked that the right wing media has not reported more about the "censorship" inherent to some kinds of word vector debiasing techniques proposed in papers after the famous "man is to computer programmer as woman is to homemaker?" paper within NLP. Usually the undesirable associations (programmer is "male" and homemaker is "female") are due to a representation gap either inherit to reality (far fewer female programmers) or due to failure to properly write about and represent female programmers in corpra (Wikipedia editors should write more articles about female programmers). One of the techniques proposed to prevent this bias is to simply find the specific rows in your dataset that maximizes the undesirable association and remove these rows. In practice, most of these "undesirable" rows (let's say a row is a wikipedia article) will have no inherit bias what-so-ever, and only get their poor scores because the article is about someone where the editors/authors wrote a lot about their relationship with their dad (using male words) before talking about their programming career. It leaves me with a bad taste in my mouth to remove not-sexist articles to mitigate or remove sexist associations. Maybe the OP could have critiqued this kind of work rather than their original post and it may have been more constructive...
- 6y ago

Permit 6y ago

> Only four papers were rejected because of ethical considerations, after a thorough assessment that included the original technical reviewers, the area chair, the senior area chair and also the program chairs. Is it possible to read these papers?

CJefferson 6y ago
In general the rejected papers are never discussed by conferences, as the authors will want to keep their from being public until they can publish it. I don't entirely agree with this, and some subjects are moving away from this way of working, but it is still very common in computing.
- dhairya 6y ago
  I agree as well, it's valuable to see rejected papers or at the very least the reviews themselves. It's interesting that some Neurips workshops are on OpenReview but Neurips itself is not. I've found great value in being able to read the reviews (both as a reviewer and learning how to do good reviews and as writer)
  - craigacp 6y ago
    The reviews for published NeurIPS papers for at least the past 5 years are available from each paper's page on http://papers.neurips.cc http://papers.neurips.cc.

nharada 6y ago

I for one am happy to see this. I often see the argument that technology can be good or bad depending on how it's used, and I would agree with this. However, too often the solution to preventing misuse proposed by people is "don't think about it", which doesn't seem like a good one to me.

Master_Odin 6y ago

A lot of these pieces on "ethical AI" just seems like theater to me. I'm sure if you ask the authors of the modern techniques of facial recognition, they'd tell you all about the wonderful and great things you can accomplish with it. And yet we also have state actors using it for great harm, and this is true of a lot of foundational AI work that then powers AI solutions to problems. Maybe this kind of "push" from conferences will help power a change, but the given the replication crisis (which the author mentions) has been well known for a long time and nothing has really happened on that front, I wouldn't hold my breath.

asciident 6y ago
There's a handful of ethical AI tropes I hear over and over. The (made up) story about Target predicting a pregnancy before she knew, the concern of using it to predict parole, face detection not working with dark-skinned people, and the trolley problem. Every talk is some permutation of these. I would love to hear actual solutions, guidelines, and deeper threads of research.
- naresh_xai 6y ago
  Happy to share a lot more ethical and challenges of non transparent AI via email with you. Bigger companies are pretty much hiding behind non transparent models and trying to ignore their failures at the moment.

tawtawtaw 6y ago

This is a theory-focused academic conference, so all of the work being presented is pretty fundamental, meaning it could underpin all kinds of applications. By analogy, if I invent a hammer that is balanced to drive nails faster, should I consider the broader impact that a bad actor could build a shelter faster? This is all about politicizing fundamental research, because it's in a field that's in the spotlight. As work becomes more applied, I can see the logic that scrutiny and consideration of its ethical implications should be increased, but pretending that authors of the kind of fundamental advances presented at a research conference like this need to speculate on the kind of end applications they could underpin is just people with nothing of value to contribute trying to muscle in to a popular venue.

kaitai 6y ago
I was a theoretical mathematician for a long time, and even there we had to talk about Broader Impacts for NSF grants, for instance. It wasn't really talking about applications for us, necessarily -- people touted the summer camps they'd present at or the undergrads they'd sponsor for research or the way they would give grant money to help undergrads attend conferences (always tied to the pure math in the grant). These NeurIPS Broader Impact statements seem a bit different, in that they more explicitly call out ethical considerations. It's funny though that you ask about hammers & nails. Pure math had a strong tradition of people ending up there because they did not want to do applied math, physics, or engineering that could be weaponized and used to kill people/support oppressive governments/infringe upon basic freedoms. Pure math is not particularly thoughtful about ethics, but many folks ended up there because of their ethical views. The Cold War, World War 2, Vietnam, and French occupation of Algeria had particular ripple effects in pure math because of that. It would be a shame if ML also became a field people avoided because they didn't want to contribute to evil, in their own view. Take the longer view: it's easy to think we're the first people ever to think about ethics and science/tech/math, but far from it. How are today's struggles the same or different than Grothendieck's, or Laurent Schwartz's, or Emmy Noether's, or Lagrange's or Kovalevskaya's or...? They all had significant reckonings to contend with.
- mkasu 6y ago

xiphias2 6y ago

While it's great to talk about broader impact of new algorithms, I think it's too late to limit AI from taking control over humanity. I see exponentially stronger algorithms every year, but the advancement in being able to set a limit to it is nothing so far.

croissants 6y ago
> I see exponentially stronger algorithms every year You do? I don't. My sense as somebody near (but not in) the ML research community is that, while there have been recent flashy new things like GPT3 or the new protein folding benchmark, these are more like "solid improvements to applications of things we mostly already know". Are you referring to something else?
- xiphias2 6y ago
  You can take almost any deep learning task that we had 5 years ago, and we can train the same task about 1000x cheaper because of hardware improvements that are slowing down, and algorithmic improvements that don't seem to be slowing down. Jeff Dean had an overview article about it I think. Also many people forget that we have working self driving cars on the road, 20 years ago nobody thought that it would happen so fast.

lasagnaphil 6y ago

The more important issue with ethics in AI is that there aren't that many people well-versed in both technology and philosophy (Throughout history many prominent scientists and engineers also dabbled a lot in the humanities, but today it seems there aren't that much.) Therefore, on one side many people in the humanities greatly misunderstands and either fantasizes or fear-mongers AI up to ridiculous heights (sometimes venturing into science-fiction territory), or many scientists and the engineers think there is absolutely no problem in applying algorithms that can be comparable to phrenology [1] to society. But most importantly, it seems like there needs to be a fundamental advancement in philosophy (beyond the current state of modernism and postmodernism), if we really want to criticize the negative aspects of technocratic governance. Maybe here's an idea I had for a while: I think there is a deeper lineage to machine learning that nobody is talking about: optimization. (Hot take: isn't what we're all doing with deep learning just optimization to fit large datasets?) I think in order to actually create a meaningful discussion about today's AI, it is crucial to look back at the history of optimization (all the way back into the 30s Soviet, where they tried to use linear programming to their planned economy and failed [2].) I think there needs to be a philosophical framework capable of articulating the effects of optimization technology on our society, before we can go any further and tackle the issues with today's and tomorrow's AI. [1] https://en.wikipedia.org/wiki/Phrenology https://en.wikipedia.org/wiki/Phrenology [2] https://crookedtimber.org/2012/05/30/in-soviet-union-optimization-problem-solves-you/ https://crookedtimber.org/2012/05/30/in-soviet-union-optimiz...

Fede_V 6y ago

I'd rather have a section of the conference that's clearly dedicated to ethical issues in AI - or - mandating rigorous discussion in papers where it is clearly warranted (ie, anything concerning facial recognition/generation, de-identification, etc) than asking all authors to write an additional section. NeurIPS is an extraordinarily competitive conference. Most authors will look at it as another way to potentially be rejected by a reviewer/area chair (in spite of all assurances about the process), and will write something incredibly trite and bland to make sure nobody is upset. Another concern is that this leads to a death by a thousand cuts - it's very easy to justify asking authors to write an additional section - but - if someone with sufficient authority doesn't say no, the logical endpoint is something like the application process for tenure track positions, where you need to produce something in the order of hundreds of pages (teaching statement, research statement, cover letter, diversity statement, list of funding, recommendation letters, etc).

throwawaysea 6y ago

The injection of politics and fuzzy notions of morality into science and technology is corrupting the objective truth seeking foundations of these fields. NeurIPS needs to abandon this direction immediately or else its legitimacy will be in question. This will lead to the publishing authors affixing pandering impact statements to cater to the personal politics whims of those judging their work, and will also lead them to avoid risky exploration or challenges to the political status quo. The legitimacy of AI Ethics as a subject is in serious doubt already after the Timnit Gebru fiasco, but if NeurIPS wants to invite politics into the AI world, they should invite those writing papers on ethics to analyze others’ work instead of requiring such impact statements of authors who are focused on core technology.