4 ms·

This is a theory-focused academic conference, so all of the work being presented is pretty fundamental, meaning it could underpin all kinds of applications. By

This is a theory-focused academic conference, so all of the work being presented is pretty fundamental, meaning it could underpin all kinds of applications. By analogy, if I invent a hammer that is balanced to drive nails faster, should I consider the broader impact that a bad actor could build a shelter faster? This is all about politicizing fundamental research, because it's in a field that's in the spotlight. As work becomes more applied, I can see the logic that scrutiny and consideration of its ethical implications should be increased, but pretending that authors of the kind of fundamental advances presented at a research conference like this need to speculate on the kind of end applications they could underpin is just people with nothing of value to contribute trying to muscle in to a popular venue.

kaitai 6y ago
I was a theoretical mathematician for a long time, and even there we had to talk about Broader Impacts for NSF grants, for instance. It wasn't really talking about applications for us, necessarily -- people touted the summer camps they'd present at or the undergrads they'd sponsor for research or the way they would give grant money to help undergrads attend conferences (always tied to the pure math in the grant). These NeurIPS Broader Impact statements seem a bit different, in that they more explicitly call out ethical considerations. It's funny though that you ask about hammers & nails. Pure math had a strong tradition of people ending up there because they did not want to do applied math, physics, or engineering that could be weaponized and used to kill people/support oppressive governments/infringe upon basic freedoms. Pure math is not particularly thoughtful about ethics, but many folks ended up there because of their ethical views. The Cold War, World War 2, Vietnam, and French occupation of Algeria had particular ripple effects in pure math because of that. It would be a shame if ML also became a field people avoided because they didn't want to contribute to evil, in their own view. Take the longer view: it's easy to think we're the first people ever to think about ethics and science/tech/math, but far from it. How are today's struggles the same or different than Grothendieck's, or Laurent Schwartz's, or Emmy Noether's, or Lagrange's or Kovalevskaya's or...? They all had significant reckonings to contend with.
- mkasu 6y ago
  > It would be a shame if ML also became a field people avoided because they didn't want to contribute to evil, in their own view. Both ethics and privacy considerations have recently become pretty regular at Computer Vision and Multimedia Processing conferences. One very popular object detection model (called YOLO) had the main author recently leaving the field because he got concerned about military applications using his research results. https://medium.com/syncedreview/yolo-creator-says-he-stopped-cv-research-due-to-ethical-concerns-b55a291ebb29 https://medium.com/syncedreview/yolo-creator-says-he-stopped...
  - unishark 6y ago
    Privacy is fine. It's a valuable technology all of us pay for and desire better advances in. There's an entire field in CS/math known as cryptography, which is basically a subset of privacy. Ethics, however, is a humanities field. People in different political affiliations have widely diverging views on it. It will undoubtedly be used to promote the views of one political affiliation over others. Suppose you need to create a technology that can be used for war in order to better treat cancer? Who gets to choose who lives or dies?
    - mkasu 6y ago
      While I think general awareness of ethics concerns is needed, I think it might also sometimes bias research directions in itself. I.e., dealing with ethics concerns and/or ethics committees becomes a huge additional workload in itself, so the research is prioritized to minimize dealing with it. For example, one might stop a research which might help to treat cancer but dealing with the necessary approvals for patient data makes it unfeasible. Instead you switch to a general purpose target domain where it suddenly (unintentionally) could be used for war instead, but being general purpose it does not need to be approved by the ethics committee..
    - kaitai 6y ago
      These are all hard questions, but what I personally want to avoid is humanities people (or worse, business people) making all the decisions without tech people having input. Finance is a field where there were strong consequences for the misuse of mathematics -- in particular, the use of "value at risk" as a complete and sufficient risk metric or even worse a target/KPI was something widely seen by the mathematical types around as a disaster in the making and by the business types as a great tool for doing whatever the f*(& they wanted and papering it over with math. Look where that got us. Cryptography is actually a great example. Gauss called number theory the "queen of mathematics" and several key mathematicians (Hardy) escaped there as they figured it could never, ever be used for political purposes or anything else. And then oops, cryptography comes along and it's built entirely on number theory. You never know. Humanities people know what they know, and I respect that they've done stuff; but I'm sure not going to bow out of the conversation and hand all the ethics stuff off to them. While some really dig deep, some have no idea what the actual technology can do! There was this long thread recently about Proctorio and McGraw-Hill. Is it right for ML researchers who know about the shittiness of facial recognition to simply say "yeah whatever, do whatever you want to students who are more or less trapped by this system, we won't make a peep"? It's improbably that a NeurIPS paper addendum is going to make a huge difference in that particular problem, but we can 1) practice thinking about these things in preparation for disputes we can take part in, 2) provide ideas and information for journalists, politicians, and humanities folks who'll get involved along the way, 3) develop a habit of at least talking about it. And last, NeurIPS has so many people/teams submitting that I figured it would be inevitable that more checkboxes appear on the checklist for inclusion -- thinning mechanisms always appear when necessary to slow the flow. If not this, it'd be something else.
uomopertica 6y ago
I totally agree. I published one paper at NeurIPS this year and I work in a machine learning subfield where I basically only train networks on the MNIST digits dataset. It felt pretty pointless having to write down that I had no ethical concerns as my study only deals with handwritten digits. Some other researchers in my field got more creative and wrote down how their allegedly more efficient training algorithms would actually stop global warming... that really was pretty pointless.

kaitai 6y ago

I was a theoretical mathematician for a long time, and even there we had to talk about Broader Impacts for NSF grants, for instance. It wasn't really talking about applications for us, necessarily -- people touted the summer camps they'd present at or the undergrads they'd sponsor for research or the way they would give grant money to help undergrads attend conferences (always tied to the pure math in the grant). These NeurIPS Broader Impact statements seem a bit different, in that they more explicitly call out ethical considerations. It's funny though that you ask about hammers & nails. Pure math had a strong tradition of people ending up there because they did not want to do applied math, physics, or engineering that could be weaponized and used to kill people/support oppressive governments/infringe upon basic freedoms. Pure math is not particularly thoughtful about ethics, but many folks ended up there because of their ethical views. The Cold War, World War 2, Vietnam, and French occupation of Algeria had particular ripple effects in pure math because of that. It would be a shame if ML also became a field people avoided because they didn't want to contribute to evil, in their own view. Take the longer view: it's easy to think we're the first people ever to think about ethics and science/tech/math, but far from it. How are today's struggles the same or different than Grothendieck's, or Laurent Schwartz's, or Emmy Noether's, or Lagrange's or Kovalevskaya's or...? They all had significant reckonings to contend with.

mkasu 6y ago
> It would be a shame if ML also became a field people avoided because they didn't want to contribute to evil, in their own view. Both ethics and privacy considerations have recently become pretty regular at Computer Vision and Multimedia Processing conferences. One very popular object detection model (called YOLO) had the main author recently leaving the field because he got concerned about military applications using his research results. https://medium.com/syncedreview/yolo-creator-says-he-stopped-cv-research-due-to-ethical-concerns-b55a291ebb29 https://medium.com/syncedreview/yolo-creator-says-he-stopped...
- unishark 6y ago
  Privacy is fine. It's a valuable technology all of us pay for and desire better advances in. There's an entire field in CS/math known as cryptography, which is basically a subset of privacy. Ethics, however, is a humanities field. People in different political affiliations have widely diverging views on it. It will undoubtedly be used to promote the views of one political affiliation over others. Suppose you need to create a technology that can be used for war in order to better treat cancer? Who gets to choose who lives or dies?
  - mkasu 6y ago
    While I think general awareness of ethics concerns is needed, I think it might also sometimes bias research directions in itself. I.e., dealing with ethics concerns and/or ethics committees becomes a huge additional workload in itself, so the research is prioritized to minimize dealing with it. For example, one might stop a research which might help to treat cancer but dealing with the necessary approvals for patient data makes it unfeasible. Instead you switch to a general purpose target domain where it suddenly (unintentionally) could be used for war instead, but being general purpose it does not need to be approved by the ethics committee..
  - kaitai 6y ago
    These are all hard questions, but what I personally want to avoid is humanities people (or worse, business people) making all the decisions without tech people having input. Finance is a field where there were strong consequences for the misuse of mathematics -- in particular, the use of "value at risk" as a complete and sufficient risk metric or even worse a target/KPI was something widely seen by the mathematical types around as a disaster in the making and by the business types as a great tool for doing whatever the f*(& they wanted and papering it over with math. Look where that got us. Cryptography is actually a great example. Gauss called number theory the "queen of mathematics" and several key mathematicians (Hardy) escaped there as they figured it could never, ever be used for political purposes or anything else. And then oops, cryptography comes along and it's built entirely on number theory. You never know. Humanities people know what they know, and I respect that they've done stuff; but I'm sure not going to bow out of the conversation and hand all the ethics stuff off to them. While some really dig deep, some have no idea what the actual technology can do! There was this long thread recently about Proctorio and McGraw-Hill. Is it right for ML researchers who know about the shittiness of facial recognition to simply say "yeah whatever, do whatever you want to students who are more or less trapped by this system, we won't make a peep"? It's improbably that a NeurIPS paper addendum is going to make a huge difference in that particular problem, but we can 1) practice thinking about these things in preparation for disputes we can take part in, 2) provide ideas and information for journalists, politicians, and humanities folks who'll get involved along the way, 3) develop a habit of at least talking about it. And last, NeurIPS has so many people/teams submitting that I figured it would be inevitable that more checkboxes appear on the checklist for inclusion -- thinning mechanisms always appear when necessary to slow the flow. If not this, it'd be something else.

uomopertica 6y ago

I totally agree. I published one paper at NeurIPS this year and I work in a machine learning subfield where I basically only train networks on the MNIST digits dataset. It felt pretty pointless having to write down that I had no ethical concerns as my study only deals with handwritten digits. Some other researchers in my field got more creative and wrote down how their allegedly more efficient training algorithms would actually stop global warming... that really was pretty pointless.