> Some of the conversation touched on the reviewing and publishing process in computer science. “Curious if there have been discussions around having ethics rev

> Some of the conversation touched on the reviewing and publishing process in computer science. “Curious if there have been discussions around having ethics review boards at either conferences or with funding agencies (like IRB) to guide AI research,” one person wrote. (An organization’s institutional review board, or I.R.B., performs an ethics review of proposed scientific research.) I wonder if a good solution would be simply to have a rating system for accepted papers, with ratings applied by the reviewers. Basically, you'd have two numbers: Useful rating: On a scale of 1 to 10, is this a useful system that solves a real problem to make life generally better? Problematic rating: on a scale of 1 to 10, can this system be used in ways that create new problems or exacerbate old ones? So, let's use the example from the article of a system that predicts what a person looks like from their voice. Maybe the author submits it to the conference, and the reviewers accept the paper, and give it a useful rating of 3/10 (it could be used to identify criminal suspects with low probability, maybe?), and give it a problematic rating of 8/10 (could be used to discriminate against minorities or automatically apply cultural stereotypes; could be used for surveillance, could falsely implicate innocent people for crimes, etc...). The author could then decide whether they want to continue publishing the research, or drop it. For all published papers, the ratings are listed in the table of contents of the proceedings, and the papers in each section are sorted with the most useful first and the most problematic last.