5 ms·

I see multiple top level comments spouting righteous indignation about this being patronizing to engineers/programmers, expecting too much of them, 1st order -v

I see multiple top level comments spouting righteous indignation about this being patronizing to engineers/programmers, expecting too much of them, 1st order -vs- 2nd order effects, etc. Firstly, this is for researchers/academics who are publishing in journals, disseminating ideas for new technology. Secondly, the suggestion also clearly states that the work will be reviewed NOT by judging the utility in the possible consequences, but for the thoroughness of the analysis. In practice, that means that the paper's authors can not be more sloppy than the reviewer (a peer in the field who can be assumed to have a roughly similar understanding of the technology). Reviewer feedback can point out any blind spots of the paper. That strikes me as a reasonable and concrete setup unlike the FUD in these discussions. Nobody is being asked to balance the positives against the negatives. They are simply requested to initiate a conversation about the possible negatives so that other researchers building on the idea can work towards improving those aspects! (EDIT: Most papers/presentations already talk about the possible positive impacts -- nobody holds back on that for lack of expertise. It is only fair to ask authors to be less partisan and more thorough, as responsible researchers.) Whether all engineers/programmers should take analogous responsibility in trying to anticipate the consequences of their work is a related, but different question, with different trade-offs involved -- because engineers are concerned more with building things and delivering practical benefits, while research (and academic scholarship) is typically more interested in a longer term view of things. Both are important but distinct questions. Please don't conflate the two points in this discussion.

BrandonM 8y ago
I’m not sure enough people actually read the article, though. Here’s what the person proposing this change said: > In my first ever AI class, we learned about how a system had been developed to automate something that had previously been a person’s job. Everyone said, “Isn’t this amazing?” — but I was concerned about who did these jobs. It stuck with me that no one else’s ears perked up at the significant downside to this very cool invention. Statements like that are frightening to me. It’s incredibly hard to define “negative“ in a reasonable way, and I guarantee you that someone else is going to define it in a way you don’t like. What’s the negative impact of the recent discovery of a non-quantum recommendation algorithm on the field of quantum computing? Should the young researcher have spent time considering and detailing that?
- h4b4n3r0 8y ago
  Seems like Nature is encouraging modern-day Ludditism. We have a historical perspective to reject this out of hand: workers are now much better off than in the pre-industrial period. And yes, I understand that the past does not predict the future, but that cuts both ways. As a rule doom and gloom predictions do not age well.
- rflrob 8y ago
  > What’s the negative impact of the recent discovery of a non-quantum recommendation algorithm on the field of quantum computing? Should the young researcher have spent time considering and detailing that? If we consider the goal of research to improve society, then yes, I think it's reasonable to expect a researcher to spend some time considering whether what they're doing is going to actually improve society. In biology, most funding agencies require you to take a brief (~8-10 hours of class time) course on the responsible conduct of research, even if you aren't doing any human subjects research. In the specific case you gave of the non-quantum recommendation algorithm, the negative impacts should be more or less the same as the impacts of the quantum algorithm, but with less of a barrier to entry. And even the person proposing this acknowledges: "We all agree it’s hard and that we’re going to miss tonnes of them, but even if we catch just 1% or 5%, it’s worth it."
  - QML 8y ago
    I think in the case mentioned, the implication is that advertisers would be better equipped to target us with ads – which is a fair point. But why must we burden the theorists with ethics, and not the programmers or companies who bring that theory into the real world?
- wu-ikkyu 8y ago
  Why is that statement so frightening to you?
  - BrandonM 8y ago
    Because it highlights exactly how broad the definition of “negative” can be. I grew up in a family of blue collar workers. My uncle worked at steel mill that shut down. Perhaps with some automation it could have at least stayed open. Regardless, he eventually found a new rewarding job. My dad is out of a job now, too, but that’s because he got hurt at work, multiple times. He has had back issues my entire life because he got hurt at work. The person proposing a consideration of negative consequences was “inspired” by jobs being automated away. Whether a researcher has considered the negative consequences enough will be determined by peer reviewers who have their own political leanings and potential agendas. In the end, I would expect the process to become completely subverted, without accomplishing much along the way.
Kalium 8y ago
It seems quite strong to expect "researchers disclose any possible negative societal consequences of their work in papers". This isn't just a matter of knowledge and familiarity. It requires some intensely personal moral and ethical judgments. One person's moral evil (self-driving trucks putting truckers out of work) might be another's moral obligation (getting rid of the life-shortening occupational hazards of trucking). The verbiage around fully disclosing the possible outcomes adds an extra wrinkle. Reasonable results require social and technological forecasting and weighing of what researchers thing is likely or reasonable. It's possible that the researchers we're discussing might not have extensive training in forecasting large-scale social behavior. I would submit that it's worth this effort to catch some small percent of potential negative impacts only if we can do so without excessive costs from false positives. A scenario in which an excess of false positives overwhelms the number of true positives at great cost is a very possible outcome and one that merits more than casual dismissal. If we can't predict outcomes reliably and can't agree on how to weigh them, then some might consider it wise to not predicate significant and consequential decisions based on a combination thereof. You're absolutely right. Researchers should be taking a look at the long-term consequences of their work. It's just possible that researchers should not assume that they're equipped to make long-term predictions or shape the efforts of their peers around said predictions. Hecht's heart - and yours - are absolutely in the right place. We can, and should, think carefully and deeply about the consequences of all of our choices with compassion and humility.
- ssivark 8y ago
  > This isn't just a matter of knowledge and familiarity. It requires some intensely personal moral and ethical judgments. One person's moral evil (self-driving trucks putting truckers out of work) might be another's moral obligation (getting rid of the life-shortening occupational hazards of trucking). Absolutely. We shouldn't expect the authors of one paper to decide on the utility of various consequences and alternatives. Also, those consequences don't exist in a vacuum, and depend on other assumptions. Eg: Automating jobs is possibly an ethical positive provided people don't need jobs to survive. And this provides an opportunity to kick off other lines of research and inquiry. > ...excessive costs from false positives... Could someone give concrete examples of the possible costs of "false positives"? In the picture I have in mind, papers are not accepted/rejected based on whether someone likes the consequences or not. (as stated in the article) > It's just possible that researchers should not assume that they're equipped to make long-term predictions or shape the efforts of their peers around said predictions. I think excuses based on insufficient expertise is a cop-out. There is no social science specialization with predictive models such things. The authors are expected to have the most insight into the specific aspects they are writing on, and generally educated well enough to be a functioning citizen of modern society. If they can vote, they can damn well put some thought into possible consequences. -- The way I see it, there are two goals underlying this suggestion: 1. Get a better understanding of the possible consequences of different lines of advancement, and, as a community, encourage the "positive" ones while discouraging the "negative" ones. (for some value of positive/negative, which might well be emergent only after much back-and-forth) 2. Get each and every researcher to actively think about the ethics of their research and the problems they choose to solve. Most commenters are focusing on only the first one; I think the second is equally important. If the focus was mainly on the first problem, we might choose to have editors writing editorials every few months on recent advancements in some topic. Or a conference panel discussing the same. Those will help towards the first goal, but only partially towards the second goal, since most researchers could "outsource" ethics to others.
slededit 8y ago
> Firstly, this is for researchers/academics who are publishing in journals, disseminating ideas for new technology. The implicit assumption is the engineers will use it to make something bad. The scientists must anticipate what engineers will do and document potential negative actions. At worst its useless boilerplate, at best its advertising all the negative uses. Science and Engineering are inextricably linked when it comes to impacts on society. The next step of course is to stop research if its determined someone can do something bad enough with it. Why else would you go through the effort of documenting it?
edanm 8y ago
> EDIT: Most papers/presentations already talk about the possible positive impacts -- nobody holds back on that for lack of expertise. That's not really accurate. Sure, people talk about positive impacts - but in a technical CS paper, they talk about positive technical aspects, e.g. "this new method will allow you to train ML models faster". They don't usually talk about societal impacts, which is what the article is about.

BrandonM 8y ago

I’m not sure enough people actually read the article, though. Here’s what the person proposing this change said: > In my first ever AI class, we learned about how a system had been developed to automate something that had previously been a person’s job. Everyone said, “Isn’t this amazing?” — but I was concerned about who did these jobs. It stuck with me that no one else’s ears perked up at the significant downside to this very cool invention. Statements like that are frightening to me. It’s incredibly hard to define “negative“ in a reasonable way, and I guarantee you that someone else is going to define it in a way you don’t like. What’s the negative impact of the recent discovery of a non-quantum recommendation algorithm on the field of quantum computing? Should the young researcher have spent time considering and detailing that?

h4b4n3r0 8y ago
Seems like Nature is encouraging modern-day Ludditism. We have a historical perspective to reject this out of hand: workers are now much better off than in the pre-industrial period. And yes, I understand that the past does not predict the future, but that cuts both ways. As a rule doom and gloom predictions do not age well.
rflrob 8y ago
> What’s the negative impact of the recent discovery of a non-quantum recommendation algorithm on the field of quantum computing? Should the young researcher have spent time considering and detailing that? If we consider the goal of research to improve society, then yes, I think it's reasonable to expect a researcher to spend some time considering whether what they're doing is going to actually improve society. In biology, most funding agencies require you to take a brief (~8-10 hours of class time) course on the responsible conduct of research, even if you aren't doing any human subjects research. In the specific case you gave of the non-quantum recommendation algorithm, the negative impacts should be more or less the same as the impacts of the quantum algorithm, but with less of a barrier to entry. And even the person proposing this acknowledges: "We all agree it’s hard and that we’re going to miss tonnes of them, but even if we catch just 1% or 5%, it’s worth it."
- QML 8y ago
  I think in the case mentioned, the implication is that advertisers would be better equipped to target us with ads – which is a fair point. But why must we burden the theorists with ethics, and not the programmers or companies who bring that theory into the real world?
wu-ikkyu 8y ago
Why is that statement so frightening to you?
- BrandonM 8y ago
  Because it highlights exactly how broad the definition of “negative” can be. I grew up in a family of blue collar workers. My uncle worked at steel mill that shut down. Perhaps with some automation it could have at least stayed open. Regardless, he eventually found a new rewarding job. My dad is out of a job now, too, but that’s because he got hurt at work, multiple times. He has had back issues my entire life because he got hurt at work. The person proposing a consideration of negative consequences was “inspired” by jobs being automated away. Whether a researcher has considered the negative consequences enough will be determined by peer reviewers who have their own political leanings and potential agendas. In the end, I would expect the process to become completely subverted, without accomplishing much along the way.

Kalium 8y ago

It seems quite strong to expect "researchers disclose any possible negative societal consequences of their work in papers". This isn't just a matter of knowledge and familiarity. It requires some intensely personal moral and ethical judgments. One person's moral evil (self-driving trucks putting truckers out of work) might be another's moral obligation (getting rid of the life-shortening occupational hazards of trucking). The verbiage around fully disclosing the possible outcomes adds an extra wrinkle. Reasonable results require social and technological forecasting and weighing of what researchers thing is likely or reasonable. It's possible that the researchers we're discussing might not have extensive training in forecasting large-scale social behavior. I would submit that it's worth this effort to catch some small percent of potential negative impacts only if we can do so without excessive costs from false positives. A scenario in which an excess of false positives overwhelms the number of true positives at great cost is a very possible outcome and one that merits more than casual dismissal. If we can't predict outcomes reliably and can't agree on how to weigh them, then some might consider it wise to not predicate significant and consequential decisions based on a combination thereof. You're absolutely right. Researchers should be taking a look at the long-term consequences of their work. It's just possible that researchers should not assume that they're equipped to make long-term predictions or shape the efforts of their peers around said predictions. Hecht's heart - and yours - are absolutely in the right place. We can, and should, think carefully and deeply about the consequences of all of our choices with compassion and humility.

ssivark 8y ago
> This isn't just a matter of knowledge and familiarity. It requires some intensely personal moral and ethical judgments. One person's moral evil (self-driving trucks putting truckers out of work) might be another's moral obligation (getting rid of the life-shortening occupational hazards of trucking). Absolutely. We shouldn't expect the authors of one paper to decide on the utility of various consequences and alternatives. Also, those consequences don't exist in a vacuum, and depend on other assumptions. Eg: Automating jobs is possibly an ethical positive provided people don't need jobs to survive. And this provides an opportunity to kick off other lines of research and inquiry. > ...excessive costs from false positives... Could someone give concrete examples of the possible costs of "false positives"? In the picture I have in mind, papers are not accepted/rejected based on whether someone likes the consequences or not. (as stated in the article) > It's just possible that researchers should not assume that they're equipped to make long-term predictions or shape the efforts of their peers around said predictions. I think excuses based on insufficient expertise is a cop-out. There is no social science specialization with predictive models such things. The authors are expected to have the most insight into the specific aspects they are writing on, and generally educated well enough to be a functioning citizen of modern society. If they can vote, they can damn well put some thought into possible consequences. -- The way I see it, there are two goals underlying this suggestion: 1. Get a better understanding of the possible consequences of different lines of advancement, and, as a community, encourage the "positive" ones while discouraging the "negative" ones. (for some value of positive/negative, which might well be emergent only after much back-and-forth) 2. Get each and every researcher to actively think about the ethics of their research and the problems they choose to solve. Most commenters are focusing on only the first one; I think the second is equally important. If the focus was mainly on the first problem, we might choose to have editors writing editorials every few months on recent advancements in some topic. Or a conference panel discussing the same. Those will help towards the first goal, but only partially towards the second goal, since most researchers could "outsource" ethics to others.

slededit 8y ago

> Firstly, this is for researchers/academics who are publishing in journals, disseminating ideas for new technology. The implicit assumption is the engineers will use it to make something bad. The scientists must anticipate what engineers will do and document potential negative actions. At worst its useless boilerplate, at best its advertising all the negative uses. Science and Engineering are inextricably linked when it comes to impacts on society. The next step of course is to stop research if its determined someone can do something bad enough with it. Why else would you go through the effort of documenting it?

edanm 8y ago

> EDIT: Most papers/presentations already talk about the possible positive impacts -- nobody holds back on that for lack of expertise. That's not really accurate. Sure, people talk about positive impacts - but in a technical CS paper, they talk about positive technical aspects, e.g. "this new method will allow you to train ML models faster". They don't usually talk about societal impacts, which is what the article is about.