4 ms·

> A quantum computer is not like a nuclear weapon: it’s not going to directly kill anybody This is, of course, not the only problem with nuclear weapons (and b

by scroot 7y ago

> A quantum computer is not like a nuclear weapon: it’s not going to directly kill anybody This is, of course, not the only problem with nuclear weapons (and by extension the science and technology that created them). First, there are several indirect ways that a single nuclear detonation could kill people. Second, the realization of nuclear weapons create a world in which nuclear war is possible -- one where mass detonations could alter the planet so radically that there would be innumerable indirect threats to everyone. Third, consider the way that the amassing of nuclear weapons so altered the geopolitical landscape, and what the effects were. This is precisely what the letter-writer is hinting at -- that we should be competent enough to strongly consider possible unintended consequences. Despite the prominent stream of thought on this site, no particular technology is in itself "inevitable." We have the capability, collectively, to choose whether or not to pursue something.

rcoveson 7y ago
> We have the capability, collectively, to choose whether or not to pursue something. This is the coordination problem to end all coordination problems. Is there an example in history of civilization turning its collective back on some scientific pursuit in fear of its potentially negative consequences? It seems close to impossible. There are too many different value structures out there, and there’s too much money on offer to anyone who can give any government or large corporation an advantage over its competitors. You may be able to convince a few researchers today, but as soon as China has the slightest breakthrough towards X or Y technology, most of those who signed the pledge will be tripping over themselves to get cracking on the problem.
- scroot 7y ago
  > Is there an example in history of civilization turning its collective back on some scientific pursuit in fear of its potentially negative consequences? It seems close to impossible. First we should ask: is this a "scientific" pursuit in the first place? Or, rather, are we describing the creation of some mechanisms (based themselves on scientific discovery) that can be used for all kinds of purposes, among them, yes, future scientific practice? Computing leverages scientific discovery, and aids in its practice, but is itself not science. Second, and at the risk of sounding trite, we can also ask "is there an example in the history of civilization of someone flying before the Wright brothers?" or "is there an example in the history of civilization of a world governing body before the 20th century?" The very promise of democracy, in general terms, is that we should, in fact, be able to make large decisions together on a society-wide scale. And because that is, ostensibly, the highest level through which we coordinate, we can should also be able to direct how our democratic institutions help us achieve those aims. If your position is that money, corporate influence, or government corruption prevent such coordination, you are certainly correct. But these are not "natural law." We have the ability -- however likely -- to deal with those problems. Otherwise, what's the point? > You may be able to convince a few researchers today, but as soon as China has the slightest breakthrough towards X or Y technology, most of those who signed the pledge will be tripping over themselves to get cracking on the problem. We are not limited to pledges. But even in that case, it is then up to those scientists who turn a dime to deal with their own potential moral failings.
notelonmusk 7y ago
> Despite the prominent stream of thought on this site, no particular technology is in itself "inevitable." And what's your argument? If the technology represents a big advantage to a group of people, history shows that is what eventually happens. So yeah we could choose not to invent spears or choose not to invent metal armor, or choose not to invent gunpowder, or...
- scroot 7y ago
  > If the technology represents a big advantage to a group of people, history shows that is what eventually happens. It depends on what you mean by "advantage" and "group of people" here. An "advantage" is more than utilitarian in most cultures, and has to fit into the ideological context in which it emerges in order to be even considered relevant. Universally we assume that writing provides an unmatched advantage in discovery and knowledge production. But many cultures who had contact with writing did not take it up. The same is true of the wheel. If you are conducting activity to appease / observe the reaction of Huitzilopochtli (Mexica) or "The Market" (American, etc) you are going to get different technical practices and likely divergent ideas about what is advantageous. > And what's your argument? That in a democratic society that is literate and at least somewhat aware of its own history (as a practice), we can and should make informed collective decisions about what we want to do. We don't have to watch how the gods of our time -- Huitzilopochtli or The Market -- react in order to do this. Insofar as we do, however, we limit the kinds of collective decisions we can make, though in no sense is that inevitable. > So yeah we could choose not to invent spears or choose not to invent metal armor, or choose not to invent gunpowder, or... Exactly. We can choose.

by scroot 7y ago

rcoveson 7y ago

> We have the capability, collectively, to choose whether or not to pursue something. This is the coordination problem to end all coordination problems. Is there an example in history of civilization turning its collective back on some scientific pursuit in fear of its potentially negative consequences? It seems close to impossible. There are too many different value structures out there, and there’s too much money on offer to anyone who can give any government or large corporation an advantage over its competitors. You may be able to convince a few researchers today, but as soon as China has the slightest breakthrough towards X or Y technology, most of those who signed the pledge will be tripping over themselves to get cracking on the problem.

scroot 7y ago
> Is there an example in history of civilization turning its collective back on some scientific pursuit in fear of its potentially negative consequences? It seems close to impossible. First we should ask: is this a "scientific" pursuit in the first place? Or, rather, are we describing the creation of some mechanisms (based themselves on scientific discovery) that can be used for all kinds of purposes, among them, yes, future scientific practice? Computing leverages scientific discovery, and aids in its practice, but is itself not science. Second, and at the risk of sounding trite, we can also ask "is there an example in the history of civilization of someone flying before the Wright brothers?" or "is there an example in the history of civilization of a world governing body before the 20th century?" The very promise of democracy, in general terms, is that we should, in fact, be able to make large decisions together on a society-wide scale. And because that is, ostensibly, the highest level through which we coordinate, we can should also be able to direct how our democratic institutions help us achieve those aims. If your position is that money, corporate influence, or government corruption prevent such coordination, you are certainly correct. But these are not "natural law." We have the ability -- however likely -- to deal with those problems. Otherwise, what's the point? > You may be able to convince a few researchers today, but as soon as China has the slightest breakthrough towards X or Y technology, most of those who signed the pledge will be tripping over themselves to get cracking on the problem. We are not limited to pledges. But even in that case, it is then up to those scientists who turn a dime to deal with their own potential moral failings.

notelonmusk 7y ago

> Despite the prominent stream of thought on this site, no particular technology is in itself "inevitable." And what's your argument? If the technology represents a big advantage to a group of people, history shows that is what eventually happens. So yeah we could choose not to invent spears or choose not to invent metal armor, or choose not to invent gunpowder, or...

scroot 7y ago
> If the technology represents a big advantage to a group of people, history shows that is what eventually happens. It depends on what you mean by "advantage" and "group of people" here. An "advantage" is more than utilitarian in most cultures, and has to fit into the ideological context in which it emerges in order to be even considered relevant. Universally we assume that writing provides an unmatched advantage in discovery and knowledge production. But many cultures who had contact with writing did not take it up. The same is true of the wheel. If you are conducting activity to appease / observe the reaction of Huitzilopochtli (Mexica) or "The Market" (American, etc) you are going to get different technical practices and likely divergent ideas about what is advantageous. > And what's your argument? That in a democratic society that is literate and at least somewhat aware of its own history (as a practice), we can and should make informed collective decisions about what we want to do. We don't have to watch how the gods of our time -- Huitzilopochtli or The Market -- react in order to do this. Insofar as we do, however, we limit the kinds of collective decisions we can make, though in no sense is that inevitable. > So yeah we could choose not to invent spears or choose not to invent metal armor, or choose not to invent gunpowder, or... Exactly. We can choose.