4 ms·

It seems quite strong to expect "researchers disclose any possible negative societal consequences of their work in papers". This isn't just a matter of knowle

by Kalium 8y ago

It seems quite strong to expect "researchers disclose any possible negative societal consequences of their work in papers". This isn't just a matter of knowledge and familiarity. It requires some intensely personal moral and ethical judgments. One person's moral evil (self-driving trucks putting truckers out of work) might be another's moral obligation (getting rid of the life-shortening occupational hazards of trucking). The verbiage around fully disclosing the possible outcomes adds an extra wrinkle. Reasonable results require social and technological forecasting and weighing of what researchers thing is likely or reasonable. It's possible that the researchers we're discussing might not have extensive training in forecasting large-scale social behavior. I would submit that it's worth this effort to catch some small percent of potential negative impacts only if we can do so without excessive costs from false positives. A scenario in which an excess of false positives overwhelms the number of true positives at great cost is a very possible outcome and one that merits more than casual dismissal. If we can't predict outcomes reliably and can't agree on how to weigh them, then some might consider it wise to not predicate significant and consequential decisions based on a combination thereof. You're absolutely right. Researchers should be taking a look at the long-term consequences of their work. It's just possible that researchers should not assume that they're equipped to make long-term predictions or shape the efforts of their peers around said predictions. Hecht's heart - and yours - are absolutely in the right place. We can, and should, think carefully and deeply about the consequences of all of our choices with compassion and humility.

ssivark 8y ago
> This isn't just a matter of knowledge and familiarity. It requires some intensely personal moral and ethical judgments. One person's moral evil (self-driving trucks putting truckers out of work) might be another's moral obligation (getting rid of the life-shortening occupational hazards of trucking). Absolutely. We shouldn't expect the authors of one paper to decide on the utility of various consequences and alternatives. Also, those consequences don't exist in a vacuum, and depend on other assumptions. Eg: Automating jobs is possibly an ethical positive provided people don't need jobs to survive. And this provides an opportunity to kick off other lines of research and inquiry. > ...excessive costs from false positives... Could someone give concrete examples of the possible costs of "false positives"? In the picture I have in mind, papers are not accepted/rejected based on whether someone likes the consequences or not. (as stated in the article) > It's just possible that researchers should not assume that they're equipped to make long-term predictions or shape the efforts of their peers around said predictions. I think excuses based on insufficient expertise is a cop-out. There is no social science specialization with predictive models such things. The authors are expected to have the most insight into the specific aspects they are writing on, and generally educated well enough to be a functioning citizen of modern society. If they can vote, they can damn well put some thought into possible consequences. -- The way I see it, there are two goals underlying this suggestion: 1. Get a better understanding of the possible consequences of different lines of advancement, and, as a community, encourage the "positive" ones while discouraging the "negative" ones. (for some value of positive/negative, which might well be emergent only after much back-and-forth) 2. Get each and every researcher to actively think about the ethics of their research and the problems they choose to solve. Most commenters are focusing on only the first one; I think the second is equally important. If the focus was mainly on the first problem, we might choose to have editors writing editorials every few months on recent advancements in some topic. Or a conference panel discussing the same. Those will help towards the first goal, but only partially towards the second goal, since most researchers could "outsource" ethics to others.
- Kalium 8y ago
  > Eg: Automating jobs is possibly an ethical positive provided people don't need jobs to survive. Or maybe automating certain jobs is an ethical positive because means fewer people die in coal pits. You're very right that this is an excellent opportunity to kick off other lines of research and inquiry. It's possible that what is essentially a journal of applied mathematics might not be an ideal venue for such, given that journals of ethics and economics already exist. > Could someone give concrete examples of the possible costs of "false positives"? In the picture I have in mind, papers are not accepted/rejected based on whether someone likes the consequences or not. (as stated in the article) What would be the point of a negative social consequences disclosure requirement if there were no consequences whatsoever for leaving one off or being glaringly silly in the disclosure? The point of this proposed process is that it is ground to reject a paper and require further revision. To put it another way: the process proposed will either have false positives or it will do nothing. The extent to which it has false positives that exact a cost will be determined by the extent to which the process works as intended to discourage research deemed to have negative social consequences. Rejecting a paper might end a grad student's career. It might prevent disclosure of a significant finding in time to have a major impact on something important. We have no way of anticipating the consequences of this. Perhaps this proposal should come with a discussion of the potential negative social consequences of it, given that some can already be reasonably foreseen. Given how abstract that is, I shall offer a more concrete example. My past involves research into how to make circuits more efficient and easily produced reliably and at scale. At no point in this process did anyone stop and think about if this might have applications in computers used as part of weapons systems. Which, looking back, is obviously one application of computers. > The authors are expected to have the most insight into the specific aspects they are writing on, and generally educated well enough to be a functioning citizen of modern society. You're absolutely right. The authors can be expected to have deep and significant insight into the technical or mathematical aspects of their work. It's possible that this might not be the same as having the most insight into economic or social consequences of their work. > 1. Get a better understanding of the possible consequences of different lines of advancement, and, as a community, encourage the "positive" ones while discouraging the "negative" ones. (for some value of positive/negative, which might well be emergent only after much back-and-forth) This is a laudable goal! And it's perhaps worth considering if there might be unintentional and perhaps "negative" side effects to such a process. > 2. Get each and every researcher to actively think about the ethics of their research and the problems they choose to solve. If this is the goal, then perhaps the requirement should be for every paper to be accompanied with an ethical statement which is in no way part of the review process. That would satisfy your needs without risking significant unintentional side-effects, would it not?

by Kalium 8y ago

ssivark 8y ago

> This isn't just a matter of knowledge and familiarity. It requires some intensely personal moral and ethical judgments. One person's moral evil (self-driving trucks putting truckers out of work) might be another's moral obligation (getting rid of the life-shortening occupational hazards of trucking). Absolutely. We shouldn't expect the authors of one paper to decide on the utility of various consequences and alternatives. Also, those consequences don't exist in a vacuum, and depend on other assumptions. Eg: Automating jobs is possibly an ethical positive provided people don't need jobs to survive. And this provides an opportunity to kick off other lines of research and inquiry. > ...excessive costs from false positives... Could someone give concrete examples of the possible costs of "false positives"? In the picture I have in mind, papers are not accepted/rejected based on whether someone likes the consequences or not. (as stated in the article) > It's just possible that researchers should not assume that they're equipped to make long-term predictions or shape the efforts of their peers around said predictions. I think excuses based on insufficient expertise is a cop-out. There is no social science specialization with predictive models such things. The authors are expected to have the most insight into the specific aspects they are writing on, and generally educated well enough to be a functioning citizen of modern society. If they can vote, they can damn well put some thought into possible consequences. -- The way I see it, there are two goals underlying this suggestion: 1. Get a better understanding of the possible consequences of different lines of advancement, and, as a community, encourage the "positive" ones while discouraging the "negative" ones. (for some value of positive/negative, which might well be emergent only after much back-and-forth) 2. Get each and every researcher to actively think about the ethics of their research and the problems they choose to solve. Most commenters are focusing on only the first one; I think the second is equally important. If the focus was mainly on the first problem, we might choose to have editors writing editorials every few months on recent advancements in some topic. Or a conference panel discussing the same. Those will help towards the first goal, but only partially towards the second goal, since most researchers could "outsource" ethics to others.

Kalium 8y ago
> Eg: Automating jobs is possibly an ethical positive provided people don't need jobs to survive. Or maybe automating certain jobs is an ethical positive because means fewer people die in coal pits. You're very right that this is an excellent opportunity to kick off other lines of research and inquiry. It's possible that what is essentially a journal of applied mathematics might not be an ideal venue for such, given that journals of ethics and economics already exist. > Could someone give concrete examples of the possible costs of "false positives"? In the picture I have in mind, papers are not accepted/rejected based on whether someone likes the consequences or not. (as stated in the article) What would be the point of a negative social consequences disclosure requirement if there were no consequences whatsoever for leaving one off or being glaringly silly in the disclosure? The point of this proposed process is that it is ground to reject a paper and require further revision. To put it another way: the process proposed will either have false positives or it will do nothing. The extent to which it has false positives that exact a cost will be determined by the extent to which the process works as intended to discourage research deemed to have negative social consequences. Rejecting a paper might end a grad student's career. It might prevent disclosure of a significant finding in time to have a major impact on something important. We have no way of anticipating the consequences of this. Perhaps this proposal should come with a discussion of the potential negative social consequences of it, given that some can already be reasonably foreseen. Given how abstract that is, I shall offer a more concrete example. My past involves research into how to make circuits more efficient and easily produced reliably and at scale. At no point in this process did anyone stop and think about if this might have applications in computers used as part of weapons systems. Which, looking back, is obviously one application of computers. > The authors are expected to have the most insight into the specific aspects they are writing on, and generally educated well enough to be a functioning citizen of modern society. You're absolutely right. The authors can be expected to have deep and significant insight into the technical or mathematical aspects of their work. It's possible that this might not be the same as having the most insight into economic or social consequences of their work. > 1. Get a better understanding of the possible consequences of different lines of advancement, and, as a community, encourage the "positive" ones while discouraging the "negative" ones. (for some value of positive/negative, which might well be emergent only after much back-and-forth) This is a laudable goal! And it's perhaps worth considering if there might be unintentional and perhaps "negative" side effects to such a process. > 2. Get each and every researcher to actively think about the ethics of their research and the problems they choose to solve. If this is the goal, then perhaps the requirement should be for every paper to be accompanied with an ethical statement which is in no way part of the review process. That would satisfy your needs without risking significant unintentional side-effects, would it not?